We characterize optimal taxation of foreign capital and optimal sovereign debt policy in a small open economy where the government cannot commit to policy and seeks to insure a risk averse domestic constituency. The expected tax on capital is shown to vary with the state of the economy, generating cyclicality in investment and debt in an environment where the first best capital stock is a constant. The government's lack of commitment induces a negative correlation between investment and the stock of government debt, a "debt overhang'' effect. If the government discounts the future at a rate higher than the market, then capital oscillates indefinitely at a level strictly below the first best. Debt relief is never Pareto improving and cannot affect the long-run level of investment. Further, restricting the government to a balanced budget can eliminate the cyclical distortion of investment.