In the latest episode of To Be Continuous, Edith and Paul discuss the challenges and benefits of refactoring monolithic applications into microservices. They examine various approaches for creating microservice boundaries and dispel the myth that they should be defined as small as possible.
This is episode #38 in the To Be Continuous podcast series all about continuous delivery and software development.
We all know beta testing is important—not just for understanding your customers’ needs, but also for stability and security. Every time you do a launch you are essentially asking: “Are there bugs? Is there feedback?” Both with the goal of making your product better.
Testing in production will give you the most information about the success of your new functionality. And because feature flags help separate deployment from release, they make such testing safe and easy. When it comes to beta testing, a lot of the top companies tend to adhere to a similar paradigm—test early, test often, and do it in your production environment.
So how do companies have smooth and simple transitions from alpha to beta testing, and then to full rollout? Read on to learn how top companies are approaching their beta testing using deployment tools with feature flags providing links out to more in-depth descriptions.
But before we get started, here’s a quick terminology review. Pete Hodgson refers to this use of feature flags for betas as “permissioning toggles.” Also known as a “canary launch,” this is often random like a percentage rollout. A set group, or “champagne brunch,” releases to internal users or another section or group.
6 Approaches to Product Launching
#1 Facebook is the prime example of dark launching. Their release management has to be impeccable to operate at such massive scale. Their betas are often up to million users or more.
“Although we push to production only once a week, it’s still important to test the code early in real-world settings so that engineers can get quick feedback. We make mobile release candidates available every day for canary users, including 1 million or so Android beta testers.”
#2 Hootsuite gives a typical rollout pattern for its features—starting internally and then slowly exposing to a larger audience.
Push new code then:
– Dark launch to yourself or your team to test
– Launch to the whole Hootsuite organization
– 10% of all users
– Watch graphs
– Simple means of rollback if necessary
#3 Etsy calls feature flags “config flags,” and gives a lot of credit for their process to Flickr.
“Key system-level and business level metrics (like checkout/listing/registration/sign-in rates) are projected on screens in the office and we have a number of internal dashboards that the team uses (we mainly use Ganglia and Graphite). We also have lots of switches and knobs to help us roll features out to percentages of users and ramp them up slowly, or quickly. Features are used and tested by us here at Etsy for some period of time before they are rolled out publicly.”
#4 Beta can also apply to back-end rollouts. Instagram does canary deployments to a subset of servers, using feature flags as a continuous delivery tool. It’s important for continuous delivery to perform these tests, which are key in helping them avoid failed deployments.
#5 Niantic’s Pokemon Go betas are well known and rabidly tracked by its fans. They famously roll out by region—a field test in Japan here, a limited beta in Australia, and then something in New Zealand. Sometimes these betas for features are invite-only. Here’s a write up of how they approached the rollout of the game Ingress.
#6 GoPro released their GoPro Plus product early using feature flags. By breaking the larger release into smaller features with their own testing timelines, they were able to iterate and improve continuously. The video below walks through the technology they used and the timeline from dogfood to a “big bang” marketing announcement.
“At GoPro you can kind of tell we don’t things lightly. We want to do big announcements and we want to come out with great products…we actually had smaller features that would go out, and then go for alpha testing and beta testing along the way. Shortly after March, we actually had most of the applications done from a core feature standpoint, but we kept iterating and improving those core features that we knew we were going to launch with.”
Controlling Your Rollout Like a Boss
Did you notice some trends there? These larger companies are using beta testing to do one of the following:
Testing in production with feature flags
Ability to release early and test small functionalities before a broader release
Internal tests that easily become external canaries
As more companies start to use feature management, these incremental rollouts are not the headaches they once were. Companies can be safer and smarter with how and when they expose features to their end users.
If you want to get started with feature flagging, check out featureflags.io a resource we made for the community to learn best practices.
In the latest episode of To Be Continuous, Edith and Paul are joined by Jeffrey Snover, the inventor of Windows PowerShell. Jeffrey describes the tremendous cultural shift that was needed to shift Microsoft to a continuous delivery model.
Jeffrey recalls how he overcame the cultural challenge by finding the “coalition of the willing” and starting small. Jeffrey also recounts his inspiration behind PowerShell and talks about how today every layer of the technology stack is undergoing a revolution.
This is episode #37 in the To Be Continuous podcast series all about continuous delivery and software development.
In the latest episode of To Be Continuous, Edith and Paul examine the recent backlash that Unroll.me faced over selling user data to other companies. Edith shares her experience as Product Manager at Tripit and discusses the risks of developing a service that requires accessing customer emails.
They also examine how privacy sensitivity varies among users and has evolved over time.
This is episode #36 in the To Be Continuous podcast series all about continuous delivery and software development.
In the latest episode of To Be Continuous, Edith and Paul discuss a medley of start-up pertinent topics. Paul wonders whether the pendulum has swung too far the other way, with too many startups now glorifying failure.
They also examine ways of coping with founder guilt and dispel some common myths about life in a startup. Finally, they examine the invaluable role of the “Everything Else” person in start-ups. This is episode #35 in the To Be Continuous podcast series all about continuous delivery and software development.
In the latest episode of To Be Continuous, Edith and Paul examine how monetization approaches vary depending on the developer market you’re in.
They discuss how developer markets can be divided into four spaces and examine the companies operating in each space. They also consider the pitfalls of startups that push their product in too many directions and consider how to go about evaluating the threat of Amazon cloning your service. This is episode #34 in the To Be Continuous podcast series all about continuous delivery and software development.