We live in an era of digital abundance. Cloud computing and big data are transforming how we live and work. The rise of the cloud has reduced computing costs to historic lows, while the emergence of big data has created a world awash in useful information.
These changes are disrupting what we once considered to be competitive advantages. Businesses used to create competitive advantages for themselves based on access to superior IT or exclusive data, but now these are slipping away as advantages. Today, one of the primary differentiators for organizations in data-intensive sectors will be access to talented, data-literate and skilled workers. This is particularly true for government agencies. To improve public services with data-driven technology, they’ll need to work harder than ever to recruit, hire and retain highly skilled data engineers, data scientists and managers.
To understand the relative importance of these skills, consider the degree to which cloud architectures have commoditized computing; Moore’s Law — the observation that the number of transistors on a chip will double roughly every two years — predicted the modern digital era of smartphones, tablets and wearables. But, the recently proposed Bezos’ Law — the observation that the cost of a unit of computing power in the cloud is reduced by 50 percent every three years — predicts that the cost of computing will eventually become non-limiting for most organizations.
Cloud computing has not only slashed costs for organizations, it also has given us ﬂexibility to adapt their computing infrastructure to our changing needs. This has democratized access to the latest technologies; new entrants are now on the same footing as long-established incumbents. And when everyone has access to the same technology, talent matters.
The emergence of big data has also had a similar effect. Access to massive amounts of information has become relatively cheap and begun to erode the strategic advantage that organizations with a data monopoly might have counted on in the past. Some of these advantages will remain indeﬁnitely — Netﬂix, for example, knows more about the viewing habits of its customers than anyone else — but competitors can now mine other data sources to narrow this gap. Long story short, organizations can no longer rely only on proprietary data to stay ahead and must instead compete on talent.
Yet there are too few qualiﬁed people in the labor market, to provide that talent. McKinsey Global Institute has estimated a shortfall of 140,000 to 190,000 data scientists by 2018, and an even greater shortage of managers with the analytical skills needed in this big data world. Employment in data-intensive industries is also geographically concentrated in certain states, putting other states at a disadvantage. This challenge will be especially pronounced for government agencies, which already have a problem recruiting the best and brightest of the private sector.
But there is no easy solution to this problem. Two factors that affect employment decisions (compensation and culture) require flexible budgets and organizational change; but neither of these plays to the government’s strengths. Despite this, disruptive innovation in government is possible.
Government agencies actually do have an advantage in that many of the problems they are working on (like increasing access to affordable healthcare, improving the quality of the schools, and making cities safer and cleaner) are the types of problems that attract the sharpest of minds. They may not be able to match the pay of Silicon Valley, but working for the government offers the chance to improve the world.
In the long term, policymakers must ﬁx the workforce pipeline so that skills better match employer needs in the private and public sectors. But in the short term, governments will be in ﬁerce competition with the private sector for the best data scientists. They’ll need to use all available resources to bring in the human capital that can ensure the opportunities from the data revolution don’t pass them by.