The best tests are the ones you don't write

Test code shouldn't be some script you write for an hour, for weeks of work on functional code.

Test code needs to be just as readable, maintainable, and scalable as your functional code is. You can easily spend just as time, and more, on tests. They should be valuable time.

Just like you should avoid writing code that doesn't do anything, you should avoid writing test code that doesn't really test anything.

Are you sure you need to test that if-statement? Is it imperative to make a test for something you know will not happen logically? Does it make any more sense to test that something does happen, when you know for sure it will happen like it's designed?

The best practice for testing, is to write tests for complex components that might break. Not break because the computer will suddenly break, but break because people might make bad changes to it. Or people might give bad values. Or people might forget what that component was doing.

Tests are to cover for human error.

Computers will not surprise you, humans will.

When you see a surprising result on a computer, the only thing off is either your understanding, or someone else's implementation.

Unless you're dealing with a computer that flies above the atmosphere, is bombarded with radiation that causes a bit to mysteriously change from a 1 to a 0, and that switch statement in your code gets a value that is not expected, it's best that you don't take a heavy-handed approach to testing everything.

Tests are to protect against human failure, not computer failure. And humans can fail in a few ways:

1. They try to change the design of things without fully understanding how it works. You don't need tests to protect against this, because usually the functionality will break in obvious ways. For instances where it wouldn't be so obvious, that's a great candidate for test cases.
2. They forget why something was done a certain way. You can use tests for this, but usually a comment in the right area is more than enough proof against this. Especially if there's a review process and others can see obvious commentary that the changer was too lazy to read.
3. They didn't think of that scenario, or those particular combination of variables. Here, tests act as a stand-in for critical thinking and planning before coding. Sure, in the process of writing the tests, you figure out that there are so many more cases that you hadn't thought of, but you would've thought of them if you had created a truth table during development.
   
   Truth tables are great, but undervalued by most developers. They'd rather think of all the possible cases with just their heads (cause they're so smart), and be surprised during development, or when the bugs start coming in.

By using tools like truth tables, design patterns, and some planning and architecting, you can build solid bug-free solutions without the need for testing, about 80% of the time. Some components might need one or two tests where some tricky area might go wrong. But not every single part of that piece requires testing.

And then there'll be 20% of your application that is complex and bug-prone. This usually ends up being a core part of your application. That's the best area to focus on heavy testing.

E.g.

Here's two recent examples where tests were a good idea:

1. I had to create a phone number prettifier, which makes the phone number look pretty as the user types. And for all possible countries.
   
   Right away I could tell that there's going to be difficulty in detecting which country. The system I was working with didn't distinguish the country code from the national phone number. So the servers were spitting out the entire international phone number and letting me deal with parsing it.
   
   Sure, all numbers are "supposed" to adhere to some standard RFC protocol, but there's like a gazillion ways people can fuck that up. And there was. Those tests saved my job and the company hundreds of hours, and thousands of unhappy users.
   
   I needed unit tests desperately to make sure I've tested as many countries, with as many formats (both correct and incorrect) as possible. I need these unit tests to make sure my functional code adheres to specific outputs from specific inputs. Fundamentals of Test-Driven Development.
   
   I had to make sure that the Google library I was using to parse phone numbers was working, and future updates to it didn't change my expected results. The library turned out to be slightly different from the android version. The unit tests saved my ass when I needed to explain why android was working but iOS wasn't, and vice-versa. It's always good to legitimately blame Google for your problems.
2. With the user’s phone number and last name, I could search through their contacts and find recommendations for the user on who to share the app to.
   
   Any recommendation engine relies of weighted graphs and algorithms to bubble up the recommended object. It'd be good to have a pool of canned contacts where I know what the result should be, and to validate my work as I develop. Another fundamental of TDD.
   
   My algorithms also needs to be proof against someone (me) mucking around in it later, trying to fix some bug. Usually a bug in these kinds of things are when something should be in the recommended list, but isn't. Here, the developer has to figure out how to update the weights, or add more parameters, so that outlier gets in, while others don't get in who don't belong, and those already in doesn't get out.

Frameworks, What Are They?

Here is an example of a framework.

You (on the left) want to reach your users (on the right), across a great chasm. The scaffold at the middle is your starting base framework. It's the System.out.println of Java. It's the Cocoa framework that iOS provides. It's Node.js for your backend servers. It's the Elastic Load Balancer on your AWS stack.

So when using this framework to complete the chasm, there are two methods. You can either,

1. Use it, or
2. Build on top of it.

Using the framework looks something like this:

You have accomplished your goal of reaching your users. But it's obvious that any extra stress on this structure will collapse. See my post on why managing stress and scale is essential to growing your product and company.

It's appropriate to use a framework for

• hackathons, and
• prototypes or proof-of-concepts.

Honestly that's the only two valid reasons I can think of for straightforward use of a framework.

"But what about agile development of MVP (Minimum Viable Product)," you ask?

I'll explain more when discussing building on top of frameworks, which looks something like this:

Notice that when you build on top of a framework, you're using the same pattern of structure that the base framework is composed of. In fact, you are extending the base framework, and building a framework of your own. 

You've added a little bit of compound interest to the world of software. If you do it right, this can be used by other people for similar chasm crossing type of problems. In fact notice the simple bridge that finally spans the chasm. If someone with a similar problem, but a wider chasm, they can copy the bridge but extend it further.

Also notice that this structure is more stable. It can bear load without breaking. It can scale without having to be torn down and re-built if the chasm were to increase, or the cliff become higher. Which will happen if you are expecting to be successful.

What about MVP? 

As the product owner, how can you expect your developers to spend all their time and energy building these structures? You need to move FAST, you don't have time for these shenanigans. 

The fallacy you're making is that it takes more time to build these structures. 

Remember that you're re-using already existing patterns from the base framework. So developers don't have to think about new ideas and new solutions, which will come with new problems and new corner cases. A developer experienced in frameworks and common design patterns can build one of these just as fast as the developer who just uses the framework.

In fact, I will attest with experience that building on top of a framework is FASTER because you will save time in the future from de-bugging and re-building when situations change.

Common framework mistakes

You have heard of the term over-engineering, Here is a depiction of what over engineering may look like:

If just using a framework is too cold, and building on top is just right, then over-engineering is too hot for Goldilocks to handle.

Some reasons that lead to over-engineering are egotism and impatience.

When I was a young developer, it was boooring to use the old tried-and-tested methods. Even as an experienced developer, the ego creeps up, and I think "I could integrate this common library to solve this problem that's been around since the beginning of time... OR I can build my own! Wouldn't that be fun??" And the answer is usually NO. Usually though, taking these detours is a great source of expanding my skills and knowledge as an engineer. But still, for the sake of your business, limit these meanderings.

Sometimes, I just don't want to spend time reading documentations and blogs on a topic. I'm ignorant of being ignorant. So I'm just going start doing something without researching if there's a solution already out there.

There's also a lack of appreciation for simplicity. It's an ego boost to be able to create something complex, something hard to understand for anybody else without my "great" mental aptitude. Who else would have thought to solve that particular problem with a frigging catapult?? The answer is no one who's experienced.

Compound interest and the Information Revolution

There's a well known urban legend about Einstein saying that compound interest is the most powerful force in the universe. Even if he might not have said it, I think he probably did appreciate the power of incremental growth. There are other common phrases like "standing on the shoulders of giants" that illustrate the same point.

Frameworks are the compound interest of programming.

A framework is a vague term in the software industry. So is a library, API, module, node, package, and so forth. They're all either related, or they all mean the same thing. It depends on who's talking and how they're using the terms.

The general idea is that a framework is something used to solve your problem, and most likely you didn't build it. For programmers, it's a relief to not have to build something, not because we're lazy, but because we know we'll just mess it up the first time. There are frameworks/libraries/packages out there that are old and reliable, all the bugs have been hashed out with use.

Frameworks are the reason why software has become such a world shifting power. In the past half century, electrical and software engineers have been incrementally growing the seeds of ideas such as Unix, HTML, Javascript and hundreds of other components. Now some of these have matured and offer tremendous leverage, allowing one person now to do the work of 100 people ten years ago, and the pace of growth is also speeding up. 

A bit of logic: How computers works

There was a course in my freshman year that we called the weed class. Unlike what it may sound like, this was the unfortunate opposite. The class was notorious for weeding you out of software engineering.

What made this class hard wasn't in the name, Digital Design Fundamentals, an introduction to boolean algebra and circuit design. It was hard because it was the first time I was made to understand how a computer works.

Little bits

You've heard that bits, 0 and 1, are how information is stored in a computer. But how do 0's and 1's translate to this application you're using to read this right now? While being useful as information in the static form, it's when you move bits around, make them dynamic, that you're able to do all the things we've been able to accomplish in the last half of the 20th century.

How are bits made dynamic? Through the use of boolean algebra. Which sounds very scary, and it is... or is it? Boolean algebra is also why information is stored in computers as bits.

It's all very logical

Luckily, I was cut out to be a software engineer, barely. Digital Design Fundamentals was the first C of my college career, one of the best C's I've ever gotten, full of great lessons. It wasn't until the last semester of my super senior year that I took the class that should've been Digital Design's pre-requisite: Introduction to Logic, which was actually an introductory course for law students.

Boolean algebra is a mathematical way of representing and solving logical problems. Boolean algebra uses ANDs, ORs, and NOTs as operators, just like you would use addition and subtraction to operate on numbers, instead in boolean algebra you operate on statements.

You are actually very familiar with logic. Not only are AND/OR/NOT operations the building blocks of computing, it's the building blocks of rationality. When you have a conversation with someone, you take turns talking (unless you're rude). Each time you talk, you present a package of communication. This communication is made of statements and a conclusion, or a point, derived and supported by the statements.

So you might present a piece of communication to someone as such:

1. It's a really hot day today.
2. We can become dehydrated from heat.
3. We should drink some water.

This statement can be represented as: A and B then C, or A & B -> C.

The listener of this communication may choose to evaluate this communication, which is to check if each part statement (A and B) is  true or false, and whether the conclusion (C) is valid or invalid.

For example, maybe the listener doesn't agree that A is true. To her, it's actually a cold day, A is false. But B is still true, you can become dehydrated from heat. Since A isn't true though, the listener doesn't agree with the conclusion, C, that water should be drank, and so C is invalid.

This is an example of boolean algebra.

At first there seems to be just these three operators, like in mathematics you start out with addition and subtraction. But if you're adding the same thing to itself multiple times, why not derive another operator, like multiplication? Same for subtraction, with division? In boolean logic there are other derived operators: NAND, NOR, XOR.

Why bits?

Isn't it odd to choose something so limiting like 0 and 1 for a numerical system? The common numerical system of this age is the decimal system. You count from 0 to 9, then start over again by incrementing the tenths place, 10, 11, 12, and on and on. There are other numerical systems like hexadecimal, which goes from 0 to 15 (represented as 0xF) and then starts over again.

Seems like the starting over again, and keeping track of how many times you've started around again, is very tedious. So why would anyone want a system where you start over again after only 1?!

The answer to this question lies in boolean algebra. To evaluate the expression, A & B -> C, the listener had to check whether each of the statements (A, B) were true or false, and whether the conclusion was valid or invalid. That's the only information necessary to evaluate any expression. You can imagine that expressions can get really large and complex. Philosophical arguments among people can become very complex and heated, each statement building on top of previous complex statements. But independent of how large an expression gets, the only thing you need to know two things like whether the statement is true/valid, or false/invalid.

Two things, dos, double, bi... binary?

The reason that computers store information in bits, 0's and 1's, is not to to let computer geeks communicate with one another through secret binary code. The reason information is stored in bits is to be able to evaluate logical expressions using boolean algebra. 0's represent false or invalid or off or whatever negative bit you need. 1 represents the positive true, valid, on, etc.

Manipulating information

I said that the way you go from static information to a dynamic application is through the manipulation and movement of information. How is boolean algebra used to manipulate information?

The first and obvious use for a computer is to be a calculator. We know, from classical mathematics, that you can do a lot of wonderful things if you can just start with the basics of adding and subtracting. One tremendous bottleneck to human achievement is how quickly we can add and subtract things.

If you can add and subtract quicker, you can do a lot of things you weren't able to do before, like go to the moon.

The invention of computers was driven largely through this need to do fast computations. And that can be done using AND and OR logic on a binary representation of any number. Here's an example of how you do 5 + 6:

Works very similarly to when you add two large numbers together and then carry the tens place over if the sum on any column goes above 9. But in a binary number system, you carry over if you go past 1.

Now, the above diagram is lying to you, it leaves a lot of things out. You can see that they're showing you the addition operation on each column, starting from the right-most column. But there is no addition operator in computers. There is only the AND, and OR, NOT and all the derivative operators. So what's going on here?

What you're looking at is actually a representation of a process where a some logic acts on each column, starting from the right, moving toward the left. This little logic machine looks something like this:

Don't get discouraged! All you're seeing is a visual representation of a couple of statements using the AND operator, which is that big D looking thing on the bottom, and an XOR operator which is the bullet looking thing at top. The two statements are:

• A XOR B -> S, and
• A AND B -> C

This is a very elegant little machine that will add a column of bits, like the 5+6 in the earlier diagram. The A, and B is the input bits from the 5 and the 6. The S is the output, and the C is any carry-over.

Understanding how an XOR works, and fully understanding this adder machine, isn't necessary at this point. What's necessary is to understand that this can be used on each column of a bitwise addition, from right to left, like you do when adding large numbers.

As a child you're taught some simple rules like starting from left, summing everything in the column, taking the carry-over and putting it in the next column and then moving on. This adder machine represents that process.

Other simple processes can be turned into more machines, there are similar simple machines to do subtraction and multiplication and a host of other manipulation of information that is required for math to occur.

Moving information

When we consider the movement of information, it's not just about moving or copying information from one location to another. What we are interested in is the process by which this movement occurs. And, just like how manipulation of bits ended up being about simple machines, movement of information also requires simple machines.

There's quite a lot of machines that are used for movement of bits, but one particular machine is the key-stone of information movement, the latch:

This machine, made of only two NAND operators, is called a latch because Q, the output, latches onto whatever S is and doesn't change even if S changes. It won't change until R, resets the latch. Then the next time it gets another input from S, it'll latch to that again. So in essence it remembers things, and that's essential when moving or copying information, you need some temporary remembering of the information you're moving.

Build it up from here

Obviously there's so much more to all this, and I don't remember any of it anymore. Remember I got a C in the class.

What I do remember is that these machines build on top of one another. Simple AND and OR's are combined to create adders and latches, which are put together to make multiplexers and decoders. These are combined to make machines that allow you to add and subtract 8-bit numbers and text, and move or copy them from one location to another.

These are then packaged into hardware like the CPU, GPU, RAM, BUS, etc. These hardware components are then connected together on motherboards that are hooked up with input devices like mouse, keyboard, modem, and output devices like speakers, monitors, and Oculus Rift.

And all of it is just a bit of logic.

That's a pun, it's actually a lot of logic that's happening really fast.

Software Engineering 101: Why is it Engineering?

Developers make you money. Engineers save you money.

Engineering is a study of stresses and forces. It's obvious when a civil engineer designs a bridge that she has to deal with different stresses and forces. Gravity pulling it down, wind pushing it from the side, seismic activity shaking it around. If you analyze other engineering fields, it's easy to see how each deals with different stresses and forces.

So what the hell is Software Engineering? What kind of stresses and forces do programs have to deal with? Even the closest kin to it, or rather the predecessor of software engineering, electrical engineering, is an obvious study of electrical forces. Like a mechanical engineer devising a combustion system that directs the flow of gasses to move pistons, electrical engineers direct the flow of electricity through various machines and gates.

But what does software engineering control? What stresses do software engineers have to account for?

Software engineering is a study of stresses and forces on information. Software engineering is a discipline of controlling the flow of information. The stresses that a software program has to deal with are of time and scale.

Information Overload

At the most basic level of programming, you're following a road. There is ever only one road, both literally and figuratively. There's always a beginning to a software program, then you can go down the code and trace the flow of information. You use variables as references to stored pieces of information, and use if-statements and for-loops to change that information around. But unlike a real road, where the destination is the important bit, for software roads it's the journey that matters. The journey down the road transforms the information into what you want.

Let's say you're the manager of a company. You have an epiphany one night and realize that the universe is vast and possibly infinite, and you are a conscience being, probably the rarest thing in the universe. You're not alone either, there are other precious conscience beings with you, and they're helping you do things that you wouldn't be otherwise capable of doing. And yet they are getting vastly under-rewarded for being such rare and amazing collection of matter. So you decide you're going to give everyone a 10% increase in pay. The next morning you bring up your employee directory and you take a random employee and multiply his salary by 1.10, then replace his current salary on the directory.

Wonderful, but you look at your directory and you realize that you have 5,000 employees, since you're a pretty successful boss. The process of taking an employees salary, modifying it, and then replacing it is a road that can be laid down by a developer. But unlike normal roads, you want to travel this road 5,000 times with every single employee.

In a software product, there are lots of roads that changes information the way you need it to. And these roads are traversed multiple times from various different directions, and they keep growing. New roads are being laid down, old roads are being changed. But how do you know that the road you're changing doesn't cause problems for some un-suspecting traveler, one you didn't know used that road?

One aspect of software engineering is how to deal with the scale of your program as it grows in complexity. And EVERY successful program will grow in complexity, it's a guarantee. As the complexity of your program grows, so does the time it takes to execute it. Depending on what you're doing, this might not be an issue. But usually programs that automate changes to information should be really fast, usually there is a lot of information to crunch.

Scale and time are the two forces that software engineering has to contend with. And it comes in various other flavors.

Relationships

Information is never alone. The nature of information is that it's informing about something. So there's always a subject, a thing, the information is describing. Flip it around and lets look at the thing itself. Most likely any thing will have more than one piece of information attached to it. A table not only has height, but width and depth as well. What's more, subjects might be pieces of information about other subjects. That table is also made up of legs. Legs also have widths, heights, and depths. Tables might have three legs, or four, or five, or maybe just one big one in the center.

You can see that if you were to create a program called "Table Factory," you would immediately come across some challenges representing the different types of tables, different types of legs, and how they are associated with one another.

These associations, or relationships, between pieces of information is both hard to create, and also hard to manage. Lets say that your table factory produces different table tops. Each of these table tops has three different configuration of legs. And you have different types of legs. "Table A" can only use two types of legs, in all three configurations (tripod, quad, single large stem). "Table B" though, can use all any types of legs, but it can only be the traditional 4 legged table. Etc.

How will you store these information such that the table factory can assemble these tables in the right configuration? There's possibly hundreds of different types of tables you can produce with all these combinations and rules, do you make a specification for each one? That would be tedious, and there is the seed of a problem of scale.

Let's say you did the calculations and you found that there are only 94 combinations, your current supply chain offers limited types of tables and legs. That's not that hard, a couple hours and you have 94 different tables specifications for how to build each one. Then your boss comes in and says, "John, I gave you a 10% raise, I'm a good boss aren't I? Well, it seems we're doing so well that I can afford to be generous! In fact, we just got a fresh supply of a new type of table legs. Could you go ahead and make sure our factories can produce tables with these new legs? It's gonna be fantastic!" And BOOM! now you have a full blown scale issue.

And with scale, comes a stress on time. Your table factory is highly optimized, down to the nano-second. Every single piece of information has to be there just when it needs to be. When the leg-attaching machine gets a Table type 3a and legs type XZ, it needs to know how to put that thing together PRONTO. Did you store the information in such a way that you can link up all the relationships and rules to get the leg-attacher to do the right thing? No? Well too bad, you didn't deal with your time constraints properly. Now your entire assembly line grinds to a halt because of bottlenecks in the system.

There goes your 10% raise.

The Business

Those are just analogies, and they seem quite academic. Who cares if things are just a bit slow and complicated? There's no real impact as long as processors get faster and memory gets cheaper, right?

There's another flavor of the time and scale forces in software engineering, and this one hits you right in the wallet. The fastest processor, or biggest hard drive in the world, isn't going to protect you.

Software changes information. But how it changes information, is itself a piece of information. You know of it as the code. And this information becomes complex very quickly unless you deal with the stresses of scale. If you're going to make money as a software developer, you're going to have to develop something useful that people pay you money for. Your software then becomes a product. And as products become popular, more and more people want it, and in better forms. So you have to hire more developers to help you grow the product. Here you have the beginnings of an issue of scale.

Like a lot of things, software is a collaborative process. Car manufacturing is a collaborative process. But no one who's responsible for the tail of a car, goes up to the front and changes the steering wheel so that the car will turn perfectly around bends, so as to display the beautiful tail he's been working on. But that does happen in software. It HAS to happen in software, due to the nature of those high-traffic roads we already talked about.

Software is a book being written by several authors. Sometimes all at once, but usually an author picks up where another author left off weeks, if not years, earlier. And each author might have different writing styles, different values of what determines a good book, and different ideas about how a character in the book should develop. This is where your scale problem explodes in your face.

When your code becomes disjointed and hacky, you start to have problems with how well your business handles the force of time . Every change now takes 10x, 100x, 1000x longer than it used to. Your competitors are coming out with better products, faster than you. Your product is released, but comes back with all sort of user complaints and bugs. Get ready to close shop, you're business is not going to make it.

Saving $$$, keeping it professional

You can go ahead and populate your startup with the scrappy and quick developers. They'll build you products and features and get you rolling and making dem dollar bills ya'll. But once you think you're about to reach some level of success, you'll find that your operation is grinding to a halt, and your costs are through the roof. What you need is an engineer to come in and streamline your production, integrate your code, and instill processes that keep things moving and efficient. It's nuanced work, it takes lots of context and experience, it takes patience, it takes planning. Shit is hard, and that's why it's a professional field of engineering, but not a very obvious one.