Are Programmers Headed Toward Another Bursting Bubble?
Interesting thoughts on the future of human jobs and machines from this 2017 piece. – Jeff
A friend of mine recently posed a question that I’ve heard many times in varying forms and forums:
“Do you think IT and some lower-level programming jobs are going to go the way of the dodo? Seems a bit like a massive job bubble that’s gonna burst. It’s my opinion that one of the only things keeping tech and lower-level computer science-related jobs “prestigious” and well-paid is ridiculous industry jargon and public ignorance about computers, which are both going to go away in the next 10 years. […]”
This question is simultaneously on point about the future of technology jobs and exemplary of some pervasive misunderstandings regarding the field of software engineering. While it’s true that there is a great deal of “ridiculous industry jargon” there are equally many genuinely difficult problems waiting to be solved by those with the right skill-set. Some software jobs are definitely going away but programmers with the right experience and knowledge will continue to be prestigious and well remunerated for many years to come; as an example look at the recent explosion of AI researcher salaries and the corresponding dearth of available talent.
Staying relevant in the ever changing technology landscape can be a challenge. By looking at the technologies that are replacing programmers in the status quo we should be able to predict what jobs might disappear from the market. Additionally, to predict how salaries and demand for specific skills might change we should consider the growing body of people learning to program. As Hannah pointed out “public ignorance” about computers is keeping wages high for those who can program and the public is becoming more computer savvy each year.
As time goes on, specific ideas and technologies push towards the “plateau of productivity” where they are eventually automated. Looking at history one must conclude that automation has the power to destroy specific job markets. In diverse industries ranging from crop harvesting to automobile assembly technology advances have consistently replaced and augmented human labor to reduce costs. A professor once put it this way in his compilers course, “take historical note of textile and steel industries: do you want to build machines and tools, or do you want to operate those machines?”
In this metaphor the “machine” is a computer programming language. This professor was really asking: Do you want to build websites using JavaScript, or do you want to build the V8 engine that powers JavaScript?
The creation of websites is being automated by WordPress (and others) today. V8 on the other hand has a growing body of competitors some of whom are solving open research questions. Languages will come and go (how many Fortran job openings are there?) but there will always be someone building the next language. Lucky for us, programming language implementations are written with programming languages themselves. Being a “machine operator” in software puts you on the path to being a “machine creator” in a way which was not true of the steel mill workers of the past.
The growing number of languages, interpreters, and compilers shows us that every job-destroying machine also brings with it new opportunities to improve those machines, maintain those machines, and so forth. Despite the growing body of jobs which no longer exist, there has yet to be a moment in history where humanity has collectively said, “I guess there isn’t any work left for us to do.”
Commodification is coming for us all, not just software engineers. Throughout history, human labor has consistently been replaced with non-humans or augmented to require fewer and less skilled humans. Self-driving cars and trucks are the flavor of the week in this grand human tradition. If the cycle of creation and automation are a fact of life, the natural question to answer next is: which jobs and industries are at risk, and which are not?
Who’s Automating Who?
AWS, Heroku, and other similar hosting platforms have forever changed the role of the System Administrator/DevOps engineer. Internet businesses used to absolutely need their own server master. Someone who was well versed in Linux; someone who could configure a server with Apache or NGINX; someone who could not only physically wire up the server, the routers, and all the other physical components, but who could also configure the routing tables and all the software required to make that server accessible on the public web. While there are definitely still people applying this skill-set professionally, AWS is making some of those skills obsolete — especially at the lower experience levels and on the physical side of things. There are very lucrative roles within Amazon (and Netflix, and Google…) for people with deep expertise in networking infrastructure, but there is much less demand at the small-to-medium business scale.
“Business Intelligence” tools such as SalesForce, Tableau and SpotFire are also beginning to occupy spaces historically held by software engineers. These systems have reduced the demand for in-house Database Administrators, but they have also increased the demand for SQL as a general-purpose skill. They have decreased demand for in-house reporting technology, but increased demand for “integration engineers” who automate the flow of data from the business to the third-party software platform(s). A field that was previously dominated by Excel and Spreadsheets is increasingly being pushed towards scripting languages like Python or R, and towards SQL for data management. Some jobs have disappeared, but demand for people who can write software has seen an increase overall.
Data Science is a fascinating example of commodification at a level closer to software. Scikit.learn, Tensorflow, and PyTorch are all software libraries that make it easier for people to build machine learning applications without building the algorithms from scratch. In fact, it’s possible to run a dataset through many different machine learning algorithms, with many different parameter sets for those algorithms, with little to no understanding of how those algorithms are actually implemented (it’s not necessarily wise to do this, just possible). You can bet that business intelligence companies will be trying to integrate these kinds of algorithms into their own tools over the next few years as well.
In many ways data science looks like web development did 5–8 years ago — a booming field where a little bit of knowledge can get you in the door due to a “skills gap”. As web development bootcamps are closing and consolidating, data science bootcamps are popping up in their place. Kaplan, who bought the original web development bootcamp (Dev Bootcamp) and started a data science bootcamp (Metis) has decided to close DevBootcamp and keep Metis running.
Content management systems are among the most visible of the tools automating away the need for a software engineer. SquareSpace and WordPress are among the most popular CMS systems today. These platforms are significantly reducing the value of people with a just a little bit of front end web development skill. In fact the barriers for making a website and getting it online have come down so dramatically that people with zero programming experience are successfully launching websites every day. Those same people aren’t making deeply interactive websites that serve billions of people, but they absolutely do make websites for their own businesses that give customers the information they need. A lovely landing page with information such as how to find the establishment and how to contact them is more than enough for a local restaurant, bar, or retail store.
If your business is not primarily an “internet business” it has never been easier to get a working site on the public web. As a result, the once thriving industry of web contractors who can quickly set up a simple website and get it online is becoming less lucrative.
Finally, it would border on hubris to ignore the physical aspect of computers in this context. In the words of Mike Acton: “software is not the platform, hardware is the platform”. Software people would be wise to study at least a little computer architecture and electrical engineering. A big shake up in hardware, such as the arrival of consumer grade quantum computers would (will) change everything about professional software engineering.
Quantum computers are still a ways off, but the growing interest in GPUs and the drive toward parallelization is an imminent shift. CPU speeds have been stagnant for several years now and in that time a seemingly unquenchable thirst for machine learning and “big data” has emerged. With more desire than ever to process large data-sets OpenMP, OpenCL, Go, CUDA, and other parallel processing languages and frameworks will continue to become mainstream. To be competitively fast in the near-term future, significant parallelization will be a requirement across the board, not just in high-performance niches like operating systems, infrastructure and video games.
Everybody Is Learning To Code
Websites are ubiquitous. The 2017 Stack Overflow Survey reports that about 15% of professional software engineers are working in an “Internet/Web Services” company. The Bureau of Labor Statistics expects growth in Web Development to continue much faster than average (24% between 2014 and 2024). Due to its visibility, there has been a massive focus on “solving the skills gap” in this industry. Coding bootcamps teach Web Development almost exclusively and Web Development online courses have flooded Udemy, Udacity, Coursera and similar marketplaces.
The combination of increasing automation throughout the Web Development technology stack and the influx of new entry level programmers with an explicit focus on Web Development has led some to predict a slide towards a “blue collar” market for software developers. Some have gone further, suggesting that the push towards a blue collar market is a strategy architected by big tech firms. Others, of course, say we’re headed for another bursting bubble.
Change in demand for specific technologies is not news. Languages and frameworks are always rising and falling in technology. Web Development in its current incarnation (“JS Is King”) will eventually go the way of Web Development of the early 2000’s (remember Flash?). What is new, is that a lot of people are receiving an education explicitly (and solely) in the current trendy web development frameworks. Before you decide to label yourself a “React developer” remember there were people who once identified themselves as “Flash developers”. Banking your career on a specific language, framework, or technology is a game of roulette. Of course it’s quite difficult to predict what technologies will remain relevant, but if you’re going to go all in on something, I suggest relying on The Lindy Effect and picking something like C that has already withstood the test of time.
The next generation will have a level of de facto tech literacy that Generation X and even Millennials do not have. One outcome of this will be that using the next generation of CMS tools will be a given. These tools will get better and young workers will be better at using them. This combination will definitely will bring down the value of low-level IT and web development skills as eager and skilled youngsters enter the job market. High schools are catching on as well, offering computer science and programming classes — some well educated high school students will likely be entering the workforce as programming interns immediately upon graduation.
Another big group of newcomers to programming are MBAs and data analysts. Job listings which were once dominated by Excel are starting to list SQL as a “nice to have” and even “requirement”. Tools such as Tableau, SpotFire, SalesForce, and other web-based metrics systems continue to replace the spreadsheet as the primary tool for report generation. If this continues more data analysts will learn to use SQL directly simply because it is easier than exporting the data into a spreadsheet.
People looking to climb the ranks and out-perform their peers in these roles are taking online courses to learn about databases and statistical programming languages. With these new skills they can begin to position themselves as data scientists by learning a combination of machine learning and statistical libraries. Look at Metis’ curriculum as a prime example of this path.
Finally, the number of people earning Computer Science and Software Engineering degrees continues to climb. Purdue, for example, reports that applications to their CS program have doubled over five years. Cornell reports a similar explosion of CS graduates. This trend isn’t surprising given the growth and ubiquity of software. It’s hard for young people to imagine that computers will play a smaller role in our futures, so why not study something that’s going to give you job security.
This article first appeared on Medium | Predict on Nov 20, 2017.
No Comments
