Steven Hicks retweeted a great thread by Marco Rogers on “full-stack developers” a little before Christmas last year. I was starting to ponder a post on how it was getting nearly impossible to be a “full stack developer”, but the thread covered a lot more than I was originally planning to, so I want to use it as a starting point. My original idea was going discuss focusing on front-end vs. back-end development, specifically how front-end development has moved away from “be able to write HTML/CSS/Javascript (optionally using jQuery)”, while Rogers’ Twitter thread (very accurately) pointed out there’s much more to the whole “full stack” thing.
Software development is fundamentally a team sport, and like any team sport, has different “positions” that require different skills to “play.” Saying you want a “full-stack developer” is like saying “I want a basketball player who can play any of the 5 positions on the floor.” Nobody can do that…well. What makes it so pervasive is the fact that modern development is no longer siloed off into “pure” back-end, front-end, DBA, operations, etc. So you have to have some skills outside your primary expertise. To keep going with my basketball analogy, this is like a small forward who can move into power forward, or if you’re really lucky, a shooting guard. The problem is that same player can’t play all 5 positions, but “can play all 5 positions” is dangerously close to what some people are asking for when they say they want a “full-stack developer.”
Marco Rogers’s Twitter thread, and the tweet that inspired it, make a good point that the job description of “full-stack developer” doesn’t even stop at “developer.” It’s now goes right into system and network engineer, system administrator, and I’ve even seen a few occasions where it got into “designer” territory as well. Oh, and let’s just ignore the fact that a lot of “full-stack developer” roles also include “being QA” since there’s a tendency to assume that since the “full stack developer” knows how to write unit tests there’s no need for testers.
It doesn’t help matters that IaaS platforms like AWS, Azure, and GCP made a lot of operational stuff really easy for people. Combined with tools like Terraform, Chef, Helm, etc. that let you store information about your infrastructure in version control and can create and update environments for you in a scriptable manner. Now your infrastructure is code too, so why can’t the developers do everything? With the emphasis from these IaaS platforms on managed services (like Lambda functions, Kinesis, and DynamoDB just to name a few from AWS), it’s even easier to fall into the trap of saying “Let’s just get a full-stack developer and they can just do all of this.”
None of this is to say you should pick 1 thing (like back-end development) and utterly ignore everything else. There’s no room for that level of specialization anymore. The key is that thinking all developers are going to be experts in all things software engineering isn’t going to work out for you. If you’re a back-end developer, you can probably build a basic web UI for an application, and do a simple deployment to something like AWS. You can probably even do some simple database management and maintenance (updating the schema and indices, for example). Most front-end developers I’ve worked with have a pretty good understanding of how the databases they’re working with are designed and the indices available so they can make sure they’re providing the information needed to perform efficient queries, and enough about back-end systems to understand the importance of paging through large datasets and how that paging is implemented in their service. Also, I’ve never met a DBA that didn’t know at least some programming, or anyone in an operations department that couldn’t script something simple up to solve a problem.
Doing any job in the software development process will require you to learn more than just 1 area of the application and stack. There’s a difference between learning to understand, or even how to perform, other parts of the software development process and being an expert at them. Here’s an example – I’m largely a back-end developer, but I know enough basic HTML, CSS, and Javascript that I can build a simple front-end for an application if I need to. If you have a pre-existing component library and thorough documentation, I can use that to build a UI. If you want me to write a custom component for you, then you’re talking to the wrong person. Likewise, I can copy and edit an existing Helm chart and edit it to get a basic service running on Kubernetes, but if you need 1 written from scratch, you want an operations person. You get the idea – a lot of people can do some stuff outside their primary focus, but nobody is a true software development Swiss Army Knife.
Just because thinking that 1 person should be able to do everything is a fallacy doesn’t excuse developers from learning additional, related skills to whatever it is they’re normally doing. Sometimes, people are out sick, or on vacation, and need someone to cover for them for a little while. Sometimes it’s a startup that just doesn’t have the budget to hire a full-fledged team, so everyone’s wearing multiple hats. Whatever the reason, being able to bounce between different parts of the software development lifecycle is an important job skill. What’s important to remember is that it’s OK to be good at your “main thing,” OK in some areas, and just not particularly good in others. Do the best you can with what you can do, and remember that 1 person can’t do everything.
That’s the dirty secret of hiring a bunch of “full-stack developers” – it’s an assumption that you can build a whole team of people who are great at everything in software development. The reality is most teams are built with people who are good at some things, competent at others, and not particularly great at still others. The whole point of having a software development team is that you don’t have to have a group of people, each of whom can do everything. Instead, you have a group of people who between them can do anything you need them to do, with some overlap to help add to the balance.
I know there are some startups that try to emphasize the “full-stack” model because they’re trying to get the most out of their money – they can afford to hire 1 person but they still need to get an app built and out the door. The truth is that the aforementioned 1 person has strengths and weaknesses, but fortunately there are tools to help compensate for those weaknesses (Javascript frameworks and HTML templates for people not great at front-end engineering, or the AWS/GCP/Azure console for people not good at operations and scripting, or services that help run your application back-end for people not particularly good at that part of the cycle). That’ll get you by when you’re small and just trying to get the whole “minimum viable product” thing on the market for a bit, but eventually these companies will expand and round out their teams.
Think of the various roles involved in building and running software as circles in a Venn Diagram. They’re all mostly distinct except for a little bit of overlapping. Asking for a “full stack developer” is like asking for someone who represents the whole graph. The reality is closer to this – everyone falls in a circle, and generally “lives” in that circle, including being competent in the parts that overlap with the other circles. Once you leave that overlap, it’s time to get help from someone else who “lives” in that circle. That’s why companies have software development teams, after all.
The original tweet and replies touch on devops (although the term doesn’t show up till late in the replies), which triggers a persistent gripe of mine. Given the way “full stack developer” is used in this tweet thread, it deserves mentioning here -devops is not a role on a software engineering team or an individual job. Devops is an organizational structure and the associated cultural practices. The point of “devops” is that the people who “develop” software and the people who “operate” software should be working closely together, on the same team, much the same way developers and QA work on the same team in most Scrum shops (PS – if you had to adopt devops, you weren’t really a Scrum shop – Scrum requires cross-functional teams that can do everything from build to running software in production, which means operations should have been on your teams since the beginning).
The downside to that analogy is that a lot of people who insist on “full stack developers” likely question the point of QA (“why can’t the developers write tests?”). Here’s the thing – good QA is hard. Figuring out all the scenarios to test is a non-trivial skill, not to mention there are different levels of testing (unit, integration, and end-to-end, for example), and that it’s always better to have someone test software other than the person who wrote it – an outside perspective invariably think of things that the person writing the code never considered. I’m all for requiring developers to write unit tests for the code they’re committing, but you should still have other people involved making sure that the code, and the whole system, works.
This brings us to a trend that companies like Netflix and Amazon have made popular – the engineers that built the software have to run it in production. The idea is that by making the engineers who built software own it the whole way through the development lifecycle, it’ll lead to more reliable software. If engineers keep getting pestered about support issues, then they’ll fix the underlying bugs and update their processes to ensure fewer bugs make it into production. Leaving aside the fact that writing software and running it in production are 2 completely different jobs (“development” and “operations” in case you’ve forgotten), there’s also the fact that there are a ton of things that can go wrong, and a developer isn’t always the right person to ping about it.
Let’s take a simple web application as an example. It doesn’t really matter what the application does, just that a user interacts with it. Now, you could see a spate of 4XX errors. Those are generally client-side errors, which means they’re likely coming from the user directly, or a client application if you have a public API (in the latter case, odds are their application isn’t authenticating correctly). Either way, these problems are going to be fixed by the user making a change, so the person to notify about that is an account manager or customer support (let’s just send this notification by email, nobody needs to be woken up at 3 AM over this).
The next scenario is a rash of 5XX errors. These are server-side error codes, which means it’s you’re software causing the problem. Now is the perfect time to call in the software developers to deal with the problem – their code is causing this and the only solution is to have them find the problem in the code and fix the bug. Depending on the customer impact and how widespread the problem is, they may be working nights or weekends, but this is the scenario that “have the team that built the software support it too” was made for.
Lastly, you have non-software-related problems with infrastructure – permissions aren’t right in your environment, nodes aren’t booting up healthy or you’re seeing a high failure/replacement rate, servers are memory swapping, etc. These are all areas where operations people excel (in fact, if you had people who specialized in operations on your team, instead of just “full-stack developers,” you’d likely avoid a lot of these problems, but I digress). They’re the ones you need to fix these errors.
Marco’s Rogers’ tweet thread makes a good point that what a lot of developers call “full-stack developer” means “can write front-end or back-end code.” I like Rogers’ label of “web developer” or “web engineer” for this work. Unfortunately, job postings for a “full-stack developer” imply that the developer to be hired is meant to be good at everything. The problem is there’s no way way from these vague job posts to tell the difference between companies that think developers should be able to do everything, or companies that want someone skilled in 1 section of the list, competent with some others, and familiar with yet others. That I blame on poor job descriptions, but what’s dangerous are companies that overlook pieces of full-fledged software development teams because they think they can just hire an equivalent (or likely fewer) number of “full stack developers.”
Unfortunately, there’s no way to figure out who’s hiring people to fill a role on a team and just wants a little bit of overlap with different parts of the team, versus who’s hiring for a team and doesn’t have the budget to get all specialists (so they’re trying to get the most bang for their buck by getting people with some solid overlaps in their skills), versus people who expect operations people who have written some useful utilities or developers who have pushed their own services on public clouds before to step in and do everything from idea to production support, until you’re in the middle of the interview process. For anyone out there trying to find work in development, good luck trying to suss out which companies are which.