Hi Gil Tayar, Senior Software Architect at Applitools. You have ~30 of experience in startups and in Hitech, so you have a really good perspective on development and deployment processes. Tell me, what is your company’s development and testing “flow”?
"Each team is pretty independent, which is as it should be in an agile environment! First of all, and probably most important of all, we do not differentiate between development and testing. Testing, for us, is a part of development.
I checked a while back the percentage of development vs testing code, and found a 60-40 ratio between them, i.e. we write more test code than application code. (Do we do TDD? Well, no. We don’t care how you write your tests, as long as you do. It’s OK to do TDD, but it’s not obligatory.)
Why do we concentrate on that? In one word: fearlessness. We need to stop fear from entering into the development cycle. Fear of deployment and fear of refactoring are the #1 inhibitors to agile development. And the only cure for that fear is testing. Testing your code (non-manually, of course) is the only way to enable you to write the code for a feature and deploying it at the same day, without fear. And testing is the only way to enable you to refactor pieces of your code and deploy it immediately without fear."
It’s interesting that you associate Release with fear. What else do you do?
"The second most important principle is that we do not have releases, and we do not have feature branches (except for extreme cases). The trunk is always in a state that is ready for deployment, and it is deployed a few times a day. This does not mean that a developer hoards the code for a feature locally until that feature is ready. Contrariwise! He writes the code for the feature piecemeal wise, and pushes and deploys each piece, even if the feature is not yet ready."
Why is writing code in small increments so important?
"Because this means that each deploy only contains a few changes, and not big changes. Which means that there is less risk involved in deploying, and if that deploy breaks the system, less risk and time involved in rolling that change back.
After all, development is all about managing risk. Because there is always risk, even if you do testing. Small changes, deployed daily, mitigate risk."
Gil Tayar, Senior Software Architect at Applitools
Deploying a few times a day—that’s impressive! How does that reflect on sprints?
"No sprints! Features are deployed as they come out of the development oven. We manage the backlog in a simple manner, assign features/bugs to a developer, and when they’re done, we deploy. We feel that this decreases the pressure felt by the developers at the end of the sprint to ensure that they “finish the features”. Yes, we still have timelines, which the developer should abide by, but it’s OK if it’s not on the day. (And, yes, sometimes we have those hard deadlines)."
I wonder what developers think of that. Anything else we can learn from?
"The third thing we do—which superficially is more of an architecture decision, rather than a process decision—is to use microservices. People tend to think of using microservices as something that is needed for projects that need large scale. And while this is true, I believe that most other projects can benefit from this architecture. Why? Because it reflects a lot on the development process, and on the two principles above: testing and small changes deployed daily."
How are microservices connected to testing?
"Microservices are, by definition, small in nature. And because they are small, they are easy to test. And making testing easy is critical in an environment that emphasizes testing as ours does. Finally, due to the fact that the unit of deployment is a small microservice, deploying a microservice, and rolling it back if necessary, is less of a deal than deploying a big monolithic application."
So heading back to the topic of developers and testing, you said that development is interconnected with testing. How does that affect the relations of developers and testers?
"Our teams are built only of developers. They deal with all aspects of product development: from writing the code, writing and executing the tests, manual QA if that is needed, building the production infrastructure, writing the deployment scripts, and deploying to, maintaining and monitoring production.
There is no specialization: each team member is required to participate in all these aspects. Unfortunately, this means no job security… But to offset that, our developers learn a lot. And agility benefits too—it enables us to create features in a minimum amount of time, without being bound by the timelines and restrictions of other teams. If you don’t have QA, and you don’t need to throw your code over the wall to be tested, then you can write the feature in a day, and deploy it in the next. And you can do that because you wrote the tests that ensure that everything is OK, and because your change is small enough that you can allow yourself to deploy it, and roll it back if something bad happens.
And if we need changes in production infrastructure? Well, we just implement them."
Let’s talk a bit about the tools you use. Which tools do you recommend, and for what?
"I am extremely satisfied with NodeJS and with Kubernetes, and the simplicity in which I can build a microservices infrastructure with both of them continues to astound me. As for the other tools, my guiding principle has always been to choose the simplest tool that can possibly work. So, while I could have used, for example, JFrog Artifactory, to store both my npm packages and Docker images, using the standard npmjs registry and docker hub is simple and good enough for my purposes. And while I could use Jenkins for all my building and deployment needs, a combination of CircleCI and deployment scripts I store in Git is much simpler and easier to maintain.
And Applitools Eyes, for visual testing, is obviously the best there is! (What? You’re not testing how your application looks like? Shame on you!)"
What do you recommend to companies who are trying to go agile?
"The first thing I recommend: don’t turn it into a “project”. It will take its time—probably a couple of years. So don’t rush it. Trying to make it happen faster just ensures that it won’t happen at all.
The second thing: make your teams product-oriented, and mostly independent from one another. This is the most difficult to do, especially in large companies, as the immediate response to that is, “we can’t do that because we’re a big company and we want control”. But control is anathema to agility. Keep those teams independent! Let them choose their development and production environment, even if that means losing control.
The third thing: once that team is independent, insist on an automated CI/CD pipeline with no feature branches. They can develop that pipeline themselves (it’s easy to do today with all the tooling that exists), but it has to be there.
The fourth thing: developers should write their own tests, and not rely on QA gating their code before deployment. This is because they should be deploying their code daily. Insist on that, otherwise your move to an agile development process will be a hollow one."
Last but not least, please share a valuable lesson you learned.
- "Write the rollback script before the first rollback is needed, not during.
- Logging everything in an orderly way is crucial. It enabled us to troubleshoot problems in production in a nice and simple way, without the need to use a debugger on production code.
- Logging performance is crucial, and enables you to figure out where your performance problems are without any of the newfangled distributed tracing tools.
- Microservices’ benefits are also for non-internet-scale projects: testing and deployment benefit from them tremendously, regardless of their benefits for scaling."
About Gil Tayar
"My name is Gil Tayar, and I started out in Mamram in the late ‘80s. I was a teacher at the Mamram School, and taught PC courses like C, C++, Assembler, and Windows. Back then, PCs were so low-powered that the other developers laughed at me, telling me I should leave the PC and come join them on “real” computers like the Mainframes and the Minicomputers. I told them that I fart in their general direction, that their mother was a hamster, and that wait and see—PCs will conquer the world. And I was right!
This concept, of the low-end computing beating out the high-end “upscale” environments, has led my thinking in all my career, and can be seen in my choice of the NodeJS platform, which people tend to think of as low-scale compared to “worthy” platforms such as the JVM. Well, just wait and see.
Besides working in various startups before and during the bubble of the late 90’s, I co-founded my own startup in the 2000s, which somehow managed to survive the collapse of 2001 and the one in 2008. After 3 years in Cloudshare, and 3 years at Wix (designing and building what is now known as Wix Code), I spent a year searching for myself and trying to build a software development bootcamp. I finally landed at Applitools, where I am currently a Senior Software Architect (which is the title you give to software developers at my age who can’t just be titled “Software developers”), leading a really cool project involving rendering web pages on a grid of hundreds of browsers."
Applitools is on a mission to help Frontend Developers, Test Automation Engineers, DevOps Teams, and Digital Transformation Executives release, test and monitor flawless mobile, web, and native apps in a fully automated way that enables Continuous Integration and Continuous Delivery (CI-CD). Founded in 2013, Applitools uses sophisticated AI powered image processing technology to ensure that an application appears correctly and functions properly on all mobile devices, browsers, operating systems and screen sizes. Applitools has more than 300 customers from a range of verticals, including Fortune 100 companies in software, banking, online retail, insurance, pharmaceuticals, and more. Applitools is headquartered in San Mateo, California, with an R&D center in Tel Aviv, Israel. For more information, please visit applitools.com.
Learn more about Continuous Testing by joining the conversation on Slack.
Want your company to be interviewed about how you are Agile? Sign up here.
To try out BlazeMeter, put your URL in the box below and your test will start in minutes.
You might also find these useful:
Interested in writing for our Blog? Send us a pitch!