The fate of different countries is remarkably diverse. Some economies expand at a fast pace, quickly developing new sectors and introducing new products and technologies. In other economies everything goes wrong.

Neo-classical economics has always had difficulty not only in explaining such disparities in growth performance, but also in providing usable policy advice about how to achieve the high growth path. But in the last fifteen years, a large number of researchers have been engaged in developing what has become known as 'endogenous growth theory'.

Traditional growth theory emphasised the role played by the accumulation of physical capital in order to explain why economies such as the United States are now so much richer than a century ago. But the accumulation of physical capital is only one part of the story. The most striking difference between most contemporary economies and those of a century ago is that today we have much better buildings and infrastructure and faster, more efficient machines, which produce more desirable products. And all this is due to technological progress, something Solow's neo-classical model treated as exogenous and hence unexplained. This left a gaping hole in our understanding of economic growth.

Endogenous growth theory sets out to fill this gap. The new growth models stemmed originally from the work of Paul Romer, who set out to explore the conditions under which an economy could exhibit sustainable growth even though there were no exogenous increases in productivity. At the same time, substantial empirical work led by Robert Barro uncovered important empirical regularities of the growth process.

The theory emphasizes that private investment in R&D is the central source of technical progress. Private firms invest in R&D only if there exist patents and other forms of property rights protection allowing firms to appropriate the rewards to the innovation process. These patent systems create, however, a temporary monopoly by restricting the number of users of the technology. From a short-run perspective these monopoly situations are undesirable, since once an invention has been made it would be better to allow other firms to compete in the production of the new product. But this would eliminate any incentives for the private sector to engage in R&D, thus lowering the rate of economic progress.

Investment in R&D is not the only way in which private investment can contribute to the growth process. Investment in human capital is an essential ingredient of growth. Highly skilled individuals, who have undergone lengthy periods of formal schooling, are responsible for the vast majority of innovations. And the effective use of new technologies often requires high levels of human capital acquired through on-the-job-training and learning-by-doing.

But identifying these potential sources of endogenous technological progress does not help us to explain why some countries fail to grow and why many developing countries are failing to catch up.

One possible explanation is 'poverty traps' – the notion that development prospects depend on the initial stocks of physical and human capital and the level of technological sophistication. Some researchers stress that the stock of human capital might be difficult to increase in economies with an initially small stock of human capital. Countries with low literacy rates, in particular, often find it very costly to increase their stock of human capital. So each generation starts out with a low level of human capital, thereby perpetuating the low skill levels in its labour force.

But public policy appears to be a more important source of cross-country growth differentials. Policies which reduce the rewards to investment by the private sector will permanently slow down the rate of economic expansion. And there are abundant examples of these policies: high taxes, inefficient education systems, lax patent protection, poor protection of general property rights and political instability. A standard explanation for the extraordinary rates of growth achieved in Hong Kong, Singapore, South Korea and Taiwan is that these countries have experienced extraordinary productivity growth. This growth in productivity has often been attributed to clever public policy that has directed public and private investment to promising sectors of society. But recent research suggests that the high levels of investment in these countries are capable of explaining most of the growth miracles.

What is clear is that successful countries have been much more open to trade than stagnant economies, while import substitution strategies have not paid off. Open economies tend to be better at absorbing new technologies that are essential to sustainable growth. They are forced to compete with the world's most advanced countries and they can more easily adapt and develop new goods and technologies because they are constantly exposed to state-of-the-art products traded in international markets.

One of the strongest empirical relationships is between the size of the financial intermediation system and the rate of growth. The size of the financial sector may matter because financial markets play a key role in allocating capital to its most efficient use. But financial markets also promote growth in a less obvious fashion. They foster the specialization that is essential to growth and allow agents to pool risk, making them more willing to invest in the development of new technologies.

There are, of course, many questions which even these new theories of growth leave unanswered. In the next decade, we must learn more about the link between institutions, policy and growth. What is clear, however, is that our understanding of the growth process has moved well beyond the traditional neo-classical growth model in which public policy could not affect long-run growth paths. Today we know that government can make a real difference – for better or worse – to long-run growth. Scientists and engineers have created the blueprints for new goods that have generated an unprecedented increase in standards of living. It may now be the turn of economists and political scientists to draw up the blueprints for new institutional arrangements which can spread the benefits of technical progress around the world.

Sérgio Rebelo is Associate Professor of Economics at the University of Rochester and at the Universidade Católica Portuguesa in Lisbon. He is a Research Fellow in CEPR's International Macroeconomics programme.