22 Dec 2017

Launched: LaunchDarkly Streaming Architecture for .NET

What is it?

Earlier this month we released the latest update to the .NET SDK. This update included a number of enhancements, regarding the SDKs ability to propagate feature flag updates quickly and efficiently. We added support for streaming flag updates via Server-Sent Events as an alternative to polling. HTTP-based streaming is favored over polling to reduce network traffic and propagate feature flag updates faster. This also enables users to harness the power of our relay proxy with the .NET SDK.

Why did we do this?

This update significantly improves the performance of flag updates for .NET applications and enables teams to take advantage of the latest platform features. This release may also be used in conjunction with the LaunchDarkly Relay Proxy.

What does it mean for LaunchDarkly users like me?

Upgrading will not require any code changes, and the streaming strategy will be used by default. Once your code is up and running with the latest version of the .NET SDK, you’ll immediately receive the benefits, including faster feature flag updates, and less outbound network traffic. Documentation for advanced configuration options can be found here.

But I have a lot going on—what does this really mean for an enterprise customer like me?

If you’re a heavy LaunchDarkly user, with hundreds, or even thousands of SDK instances out in the wild, the changes made in the newest update will be even more beneficial. Streaming architecture unlocks relay proxy support for .NET users. With the .NET SDK set up in proxy mode, your servers can connect directly to hosts within your own datacenter instead of retrieving flag updates from LaunchDarkly’s API individually.

What should I be doing to prepare for this change? (Best practices on making the update.)

Upgrading to the most recent version is as simple as changing the dependency version of “LaunchDarkly.Client” to the most recent version in your project files and package configurations. If you’re using Visual Studio, this can be done through the NuGet package manager UI. Otherwise, the NuGet command-line interface may be used. See Microsoft’s docs for complete instructions.

As of 12/21/17, the most recent version is 3.4.0.

Is there anything else I should know?

Yes! We have more in store for our .NET SDK, including:

  • Improved performance and memory usage when storing feature flags
  • Improved logging
  • Lighter use of dependencies
  • Support for caching flag configurations via redis

What if I have questions, who can I talk to?

Reach out to the LaunchDarkly support team directly by submitting a request here.

30 Nov 2017

Launched: Personal Access Tokens

LaunchDarkly users can now create personal API access tokens. Anyone who uses our API in an automated fashion knows they must authenticate with an access token. Until recently, this meant there was one token for an entire organization or account. To meet the needs of larger teams as well as others leveraging the LaunchDarkly API we wanted to provide not only greater granularity in responsibility but also better accountability. In our original token model only admins had the ability to create and view tokens since they were afforded global access and control.

What exactly are they?

Personal access tokens are an improvement on this process. Now every user can create their own access tokens for authenticated API access. Users can scope permissions, making it easier to create different tokens for different use cases. So, if API access tied to a particular account should only have read access, now you can scope the token appropriately. Also, built-in and custom roles can be assigned to access tokens in the same way that you give a team member a role. This means that the roles-based controls you put in place for the web application are extended fully to the API access on our platform as well.

Best Practices

Now that personal access tokens are available, we recommend LaunchDarkly users migrate to using these instead of account-wide tokens. The old tokens will continue to function, but we also recommend deleting those for improved security.

In terms of using the personal access tokens, we strongly suggest creating a new token for each use case and giving them the smallest number of permissions necessary to do its job. We are proponents of the principle of least privilege and security best practices that limit your risk. After all, our goal is to help you eliminate risk, not increase it.

With personal access tokens teams can also more effectively manage accountability. With an audit log, admin no longer need to worry about figuring out who has done what with a single token. Now they can monitor all access tokens based on who created them. Tokens can never do more than the user who created them, and if the user’s permissions are downgraded, so are the token’s. When the only constant in modern infrastructure is change, it’s important to have visibility into who has done what and a durable record of when it was done.

You can read more about personal access tokens in the documentation here. And if you have any comments, you can contact us here, or make comments directly in the repository.

19 Sep 2017

OpenAPI and Transparent Process

At LaunchDarkly, we’ve put a bunch of time into making our console fast and usable, and we’re pretty proud of it.

However, we’re aware there are lots of reasons people would want to use an API to create and manage feature flags. Since we are using our API to drive the dashboard, it’s easy for us to keep everything in sync if we make changes to the API. And we wanted to make it easy for our customers to do the same.

We started out with ReadMe, which is an excellent industry standard. That let us get our documents published and dynamic. For further refinements, we went to Swagger/OpenAPI. We liked it because:

  • It’s a well-known and widely-used format
  • It allows us to generate usable code snippets and examples in many languages automatically
  • It’s easy to add context and documentation to as we go.
  • We can host it on readme.io or other places, depending on our traffic needs.

We created our REST specification using OpenAPI, and you can find our documents here: LaunchDarkly OpenAPI.

We’re still working on adding examples, descriptions, and context, but we think that the documentation is stronger and more usable already.

As always, if you have any comments, you can contact us here, or make comments directly in the repository.

21 Aug 2017

How to Comply

Everyone loves a little affirmation (Image credit: twitter)

Things we wish we had known and things we were happy that we implemented early. 

At LaunchDarkly we recently embarked on the journey to SOC Type 2 compliance. While the reasons we chose to pursue this certification were primarily business driven, the tasks and actions incorporated into the process most directly impacted our engineering operations and development teams. And due to the experience and philosophy of the founding engineering team, the actually impact of the process was minimal.

Once deciding to sally forth on SOC certification (or any security and compliance certification for that matter) you will be in a much better position if you view the criteria of the certification as a benefit (or imperative) of good business practices. If you are in search of the certification primarily as a way to sell into certain customers or verticals, you will likely be frustrated with this entire process. Security is like diversity—you need to inherently believe in the value to be successful in implementations and outcome.

“So, what are these criteria you speak of?”

Need vs. want

The principle of least privilege is a well-known concept where you only provide access to the people that actually need it. This also extends to the level of access that is given. Ultimately this distills to: view vs control. I like to start here because this is a forcing function for so many of the other criteria. When you are thinking about who should have access to each system/service, you are inclined to:

  • Define roles
  • Log activity based on user/account
  • Build review process for accounts

On-boarding each new employee gives you an opportunity to review the tools you use to run your business, question who needs access, and to what extent.

This is also a great time to introduce new employees to another key security maxim: Trust, but verify. This concept is relevant in the context of the access that accounts or component services assert are needed, as well as the way that you validate user accounts.

For services—either 3rd party or just components of a larger service—you should know what information is being stored, and where. Ideally, you are being thoughtful of the first principle of least privilege.

This is especially true of customer data. The easiest way to protect data is to limit what you collect. Second is to make sure that you are being intentional and explicit with where you store the data. Finally, you need to look at the access policy that you put around that data.

For user accounts, multi-factor authentication (MFA) or two factor authentication (2FA) significantly increases your ability to validate that accounts are only being accessed by account owners.

Another time to plan for is employee off-boarding—ideally, before you off-board your first employee. This is also a good time to review your tools and access privileges.

Context; not blame

When a single developer is working on building a service, logs are primarily useful for understanding how well things are working (or where they broke). As the number of individuals working on the system becomes larger, the ability to know who changes what becomes increasingly important.

One caveat is that as you incorporate the ability to know who, you leverage this data to build context around why things changed, rather than simply using it as a means to place blame. If one person can bring down your service, then you probably should direct blame at the architect of the system. (Unless that destructive individual and the architect are the same person ¯\_(ツ)_/¯).

“A log without account context is like a novel without characters.”

A log without account context is like a novel without characters. You can build a picture of what happened, but likely will miss why things happened. If you don’t know why, you’re unlikely to prevent it from happening again.

Built for toddlers… or failure

Failure is trivial compared to proactive destruction (image credit: Lego City)

The stability of a service is often a strong indicator of the inherent security. After all, the most common exploitations are based on overloading some resource. Thinking further about the elimination of blame from building a secure and stable service, failures should be viewed as opportunities for increased robustness. This is where building for toddlers comes in.

Toddlers are the ultimate destroyers. It is the developmental stage where everyone starts to experimentally test the laws of physics. Gravity, entropy, projectile motion, harmonic oscillation—they’re all put through a battery of tests.

Ideally, you are thinking about your service from the perspective of a parent (or guardian) that is toddler-proofing their home. Bolt things down, put breakable things out of reach, lock up the flamethrower, and embrace the fact that you will miss things. For the items you miss, have an emergency procedure in place and appropriate medical supplies on hand.

Back in the context of your service…

Write it down… or it never happened

Your code/feature is not ‘done’ until the docs are written or updated. Services require constant supervision and are never ‘done’. But if a developer builds or changes something and doesn’t write it down, then they effectively become the only individual that is able to monitor or operate the entire service (at least with all the context).

So, what should you write down? “Everything,” is the easy answer, but often not the complete one. You want to write down enough to provide context if any component starts doing something unexpected.

If you don’t know where to start, a good approach is to write down what would need to happen if your service was deleted. How would you rebuild and restart your service? How would you restore your data?

Next you can look at the impact/process of the loss of each individual component service. The important part of this is to incorporate the documentation into the development process to ensure that as your service evolves your documentation is always up to date—otherwise, the change doesn’t exist after a failure.

Great, now you know what to do next time AWS S3 needs to reboot. But, what about your customers? The next step is to write down an action plan for service interruption. Make sure you have a process and plan in place for keeping folks in the loop.

Security is not the french fries

If you are in a situation where you are looking to “add” security, you are likely going to be in good company with Sisyphus. Security needs to be a part of your foundation—it is not an “add-on”. But if you are realizing this is now—that security is a requirement for your business— you can do more than wish you had considered it sooner. It is not too late, but it is not a quick fix that you can solve with a certification.

First you need to implement the security in your foundation and process. Make sure that it is part of your culture. Once you have a culture of security and process, compliance is just providing proof of your culture.

Now… about that certificate

You don’t show up to your official Genius Book of World Records judging day having never practiced juggling 9 clubs. Same goes for when you decide to get your certification for SOC.

However, when you build a strong foundation and culture for security and compliance, then the steps to get certification are rather straightforward. You call up your friendly neighborhood SOC auditor and get a copy of their check list.

In the case of LaunchDarkly we worked with the fine folks at A-lign. After an initial conversation we retained their services to conduct an independent audit of our systems and practices.

A few months prior to the audit, A-Lign provided our team with a checklist of all the documentation and proof they would need to see when they came onsite for our assessment. This afforded us the opportunity to ensure that all of our practices were organized and in a state that could be easily evaluated.

When the time came for the audit the auditor spent three days* on-site interviewing members of the team and reviewing our practices. After the on-site visit, we were informed that we had passed the initial competence certification.

Of course, now that we have gone through the validation process for one certification, it seems like a good time to keep going for a few more. Many of the certifications have a significant overlap in requirements. They are all looking to establish trust and ensure the service provider is operating in the best interest of the customer. And it turns out, most customers define trust in a very similar way.

16 Aug 2017

Week 1: How to Put Your SOC On

Enter Your Password

What does a new engineer do during their first week at a SOC 2 Compliant startup? Write code? Maybe. Deploy code? Hopefully. Create accounts? Certainly.  Generate passwords? Ad nauseam.

After creating my task tracking and document sharing accounts, half the items I saw on my TODO lists were about creating accounts on more services. Also on my calendar was to attend training for one of LaunchDarkly’s newest initiatives: SOC 2 Compliance.

At LaunchDarkly, we maintain mission critical services for our customers (feature flags!). And for those who opt for premium services, we also store sensitive data about their clients as part of our analytics features. It is essential to our business that we protect not only access to control over customer application behavior, but to all client data we store on behalf of our customers.

After our security training, each member of my incoming class made a commitment to:

  • Create a unique password for every service. Use a password generator and a password manager!
  • Enable 2-factor authentication for every service that offers it.
  • Avoid sharing passwords and accounts with team members to keep a precise audit trail.
  • Restrict browser plugins to the minimum necessary to do your job. Those plugins can read your data.
  • Secure your laptop with FileVault and lock screens.
  • Limit connected applications with access to Gmail, GitHub and other accounts.
  • Secure customer data. (Obfuscated links don’t cut it!)

These are all great practices even if your business doesn’t need SOC 2 certification. Now to deploy some code (if I can just remember where I’ve written down my SSH key…).

25 Jul 2017

Launched: LaunchDarkly SOC 2 Certification

Providing an always-on, highly secure feature management service is core to the LaunchDarkly platform. From the beginning we have designed and built our infrastructure and practices with security and availability as a priority.

Today, we are announcing the next level of this commitment to Enterprise readiness and stability and are pleased to have achieved SOC 2 Type 1 certification.

Here are a few examples of what you can read about in the report:

  • LaunchDarkly security policies
  • LaunchDarkly logical and physical access controls
  • LaunchDarkly change management process
  • LaunchDarkly data backup and disaster recovery strategies
  • LaunchDarkly system monitoring, alerts and alarms

Protecting the data and privacy of our customers is a non negotiable aspect of what we do. Our SOC 2 certification provides you with an additional assurance that we have all the right controls in place to protect your data and ensure the availability of our service and your features.

To request a copy of LaunchDarkly SOC 2 report, please email trust@launchdarkly.com.