What do internal engineering tools look like?
How engineers engineer their engineering
Most of the stuff we’ve done here at Technically has focused on building applications for users: things like Twitter, Gmail, and Salesforce that people like us use most days. But a lot of code – potentially up to 30% of it – gets written for internal use cases, i.e. apps built by your teammates for your teammates. Those use cases span from basic tools like admin panels for updating your user information all the way to complex build tools for packaging your application before deployment.
Chances are you’ve probably used at least one of these tools in your day to day. So what different types of internal tools are there? And how do they get built?
Thanks to my former employer Retool for sponsoring this post! Retool helps teams build better internal tools faster with a drag and drop component editor, easy connections to your databases and APIs, and flexible access controls for companies of all sizes.
## Let’s start with ol’ reliable: the admin panel
Ah, the prototypical workhorse of internal tools: the admin panel.
The easiest way to think about the admin panel is as an internal interface to your production database. Recall that most applications are built from two important pieces:
A frontend of HTML, CSS, and JavaScript that lays out the visual structure and interactivity of your app, and interfaces with the backend
A backend comprising the database, API endpoints sitting in front of it, and any other data-related logic hidden from your users
That frontend is what you’re reading this post in right now if you’re on the web, and the backend is storing the content, your profile information, etc. Underneath it all lies a database, probably a relational one, storing and managing all of the data needed to run your app. It probably has a users table, a table that stores payment information, and a table for the canonical unit of your app (a tee shirt listing, a post on Substack, etc.).
The frontend for whatever app you’re using – be it Substack or Gmail – is designed around specific actions that users need to take. For Substack, those actions might be:
Loading a post
Subscribing to a newsletter
Searching through a newsletter’s archive
Changing your credit card information
These are the things you want your users to be able to do, and so the frontend / APIs of your app are designed to facilitate said things. Loading a post reads post data from the DB, subscribing to a newsletter writes a new row into the DB, etc.
In other words, your users need to interact with your database, but in very specific and limited ways. We don’t want them to be able to edit the information of other users, or see another user’s credit card info.
But that set of things that your users need to do is very different from what other teams at your company might need. Your customer support team needs to be able to refund orders, see a user’s order history, change their address, etc. In other words, they need to interface with the database in different ways than your users, and this is where admin panels come in.
An admin panel is an internal frontend that lets your team members interact with your app’s database, usually to accomplish operational tasks like refunding orders, fixing bugs, changing profile information, seeing up to date sales numbers, etc. Here’s an example of an admin panel that lets a support team view and search users and orders, add a new one to the database, and edit their information:
This is only accessible to internal team members, so you’d never be able to see or use this as a user of the app.
🧠 Jog your memory 🧠
Authentication is how you identify yourself to apps and software around the web. If you’re wondering how one might authenticate for an internal tool like this: it could either be gated behind username and password at a publicly accessible URL (e.g. adminpanel.substack.com), or could be hosted on a company’s internal servers inaccessible to the public, and you’d need to be on the company’s VPN to access it.
🧠 Jog your memory 🧠
Pretty much every company in the universe has an admin panel like this if they’ve got users. At DigitalOcean we had a giant admin panel called Atlantis (everything was nautical themed) with literally thousands of actions you could take for each user (shut down their account, give them a discount, etc.).
## Different types of internal tools
Though ubiquitous, admin panels are far from the only type of internal tools you’ll see engineers build. Command line tools, buttons to kick off scripts, a CI/CD server, you name it – these are all within the fray. I’ve found that a few basic distinctions can help break down all of the different tools you could reasonably expect to see:
### For functional teams vs. for engineering
Some internal tools are built by engineers for non-engineers, while others are built by engineers for themselves. Admin panels are usually built by engineers for non-engineers, who can’t interface directly with the database (i.e. don’t know SQL), and even if they could, want a cleaner interface for doing common admin tasks. Though the nomenclature is far from precise, another good example of a tool like this is a basic dashboard that shows the number of active users on a given day.
### UI driven vs. CLI driven
Engineers build tons of internal tools for their own teams, too, and most of those tools will often not have a graphical user interface. The definition of “tool” has some nuance here, but a few examples:
CI/CD servers, tests, and workflows
Scripts for building development environments
Tools and workflows for access controls
As companies get more advanced: ML model serving platforms, internal Kubernetes interfaces, etc.
You use these tools via the command line, running scripts, etc. as opposed to navigating to a URL and seeing a bunch of buttons and tables. The bigger the company gets and the more engineers they have, the more of these kinds of tools will start popping up.
### Data oriented vs. workflow oriented
The admin panel we looked at above is just an interface on top of a company’s database, following that good old frontend/backend paradigm we’ve talked about so much here at Technically. Other types of related internal tools like dashboards follow the same model.
But not all internal tools are just reading from and writing to a database – many relate to workflows, where you need to make a few “things” happen in sequence. For example, you may have read the nicely formatted bulleted list above and wondered “what the hell is a script for building a development environment?” – and you of course trusted me to explain later. Well now it is later:
Developers work on the app and build new features on a local (on their computer) copy of the app, the actual live version of which lives in the cloud
To get that local app up and running, there’s a good deal of configuration involved
E.g. spinning up Docker containers, installing packages, making sure everything has the correct version, running tests
At some point in the early company lifecycle, developers will package all of the requisite commands for the above into a single, nice file that you can run, making this all a lot easier
Is a script with 8 commands in sequence really a “tool?” And if not, what is a “tool?” Who can say for sure – certainly not me.
## What makes internal tools different from external ones
If you’ve gotten this far you’re probably wondering why we’ve spent an entire issue of Technically talking about internal tools, and, my dear reader, your curiosity is valid. The answer is that (a) tons, probably somewhere around 30%, of developer time gets spent on these tools, and (b) they carry very different constraints around how they’re built, maintained, and measured.
I’ve heard it said that building software is a feedback loop consisting, roughly, of something like:
We build something → we see what impact it has → we use that information to build something else → and so on and so forth
With internal tools, the mechanics of this loop are all out of whack relative to external facing applications.
### Who the user is
The users of normal applications are external to the company, regular old joes like me or you or your roommate or your mom’s new friend Gary who seems to be around the house a lot. For internal tools, the users are, you guessed it, internal. This changes how the tools get built:
Smaller user base → more qualitative feedback, less quantitative
In theory, less money at stake → fewer resources (more on this below)
Usually no product managers → engineers working directly with users
No alternatives → your internal users must use the tool
For these and other reasons, internal tools can sometimes get a bad rep internally, i.e. as an engineer you’d hate to get stuck building internal stuff when you could be building external stuff. There are a lot of exceptions to this – especially at larger companies, where internal tools can be groundbreaking and interesting stuff – but I’ve found it generally holds.
### Prioritization
Without a direct tie to growth and revenue, internal tools often struggle to get the same level of resourcing, care, and maintenance as your core, external application does. If a company only has 120 engineering hours a week to allocate, it’s more likely that time goes to your actual app than the janky tool your customer support team uses to refund orders.
This creates a (sadly) vicious cycle where internal tools are often the bane of existence for teams that use them. Poorly designed UIs, buggy functionality, and slow updates are the norm, not the exception; because engineers are usually working on something else.
### Measuring success
If I build a new feature and 75% of my existing user base starts using it immediately, that's a decent signal (but not a perfect one) that building the feature was a good idea. And maybe I’ll get promoted, and finally be able to afford an apartment with a bathroom!
But how do you measure the success of your internal tools? Your internal users often have no choice but to use it, as there are no alternatives. Teams use proxies like NPS, qualitative research, or even amount of time spent using the app. None are perfect.
### Frameworks and libraries
Generally (very generally), engineering teams want their user-facing applications to be built with the most cutting edge technology and frameworks – faster, more reliable apps means happier users. Today that probably means some combination of React or Vue on the frontend, Node / JavaScript on the backend, and maybe Rust or Go if you’re feeling fancy. Even if the initial application was built years ago on something older, newer parts will be built with newer frameworks, and sometimes teams will even migrate the entire app to a newer set of technology.
For internal tools though, because of the resourcing and priority constraints, they’re often stuck on older tech like PHP, or whatever was around when the tool was built. There are also a few internal-admin specific frameworks tied to web frameworks like DjangoAdmin that (in theory) provide admin panel-like functionality out of the box. Low code solutions like Retool are making it easier for engineers to build these tools faster without needing to set everything up from scratch.
For more information and stats about how internal tools get built and what engineers think of them, I’d highly recommend checking out Retool’s survey and report here. I wrote the first draft :)
loved the extra humour in this post :)
👏🏼👏🏼👏🏼