What are Chrome Variations?

Key Takeaways

[Music] you might have heard of chrome variations or chrome field trials or occasionally even our internal code name finch well these are all names for the same thing a mechanism for chrome to do testing of new browser features and new features for chrome os which is the chrome operating system that runs on chromebooks chrome variations enable chrome to activate a new feature deactivate a feature or to try out a change to a feature for a subset of users and by feature well i just mean anything in the way chrome works and that could be a performance enhancement to the application code a change to the way the browser looks or functions or occasionally even a change to a javascript api now not everything in chrome is part of a chrome variations experiment but chrome variations can be used whenever we need to be especially careful about making a change or where a change might affect performance in ways we didn't expect chrome variations can also be used to validate a hypothesis about a change or an update for example for a percentage of users in a chrome variations experiment we tried tweaking quick networking protocol parameters to make chrome faster for users real network conditions now we've also done omnibox experiments around how many results to show another area where you may encounter chrome variations is if you work with chrome origin trials and if you're not sure what i mean by origin trial well you know check out the video about chrome's origin trials in this same series now by default an origin trial feature is enabled on all pages that provide a valid trial token in some cases however chrome variations is used to control activation of an origin trial feature and this means that an origin trial feature may not be available to certain users even when they visit a page that provides a valid trial token now information about the proportion of users eligible for activation of an origin trial feature is provided with the documentation and the updates for each origin trial so how do chrome variations work well every 30 minutes on desktop or mobile or each time you start chrome it makes a request to the chrome backend service to get the chrome variations configuration file and that's known as the variations seed when the chrome on your device contacts the chrome variations backend server to get the chrome variations configuration file it also provides information about the version of chrome and the operating system it's running on and that file returned by the chrome variations server is like a settings file to turn on features turn off features or specify variations in a feature in other words the configuration file from the chrome variations server can be used to activate deactivate features in the browser for a subset of users your browser on your device keeps track of which experimental group it belongs to chrome engineers can configure the chrome variations server so that different users get a different configuration file and that enables testing to compare performance or other metrics between users that have a feature turned on and users that have the feature turned off now one of the most important uses for chrome variations is to gradually roll out changes or new features to a percentage of users and that's crucial for a complex application like chrome that has billions of users on thousands of devices you know multiple platforms and dozens of different languages and millions of different kinds of apps and websites now of course we get valuable feedback from usage and testing in chrome canary dev and beta and we can use chrome variations to activate or deactivate features within any of these channels however those chrome channels you know are mostly used by developers and other specialists chrome stable users use chrome differently at much larger scale so we always need to test and iron out any problems encountered by normal browser usage chrome engineers can't necessarily predict how real chrome users will use chrome chrome variations is a crucial mechanism to help with this making it possible to check usage and metrics for chrome stable with a feature either activated or deactivated by measuring the impact of experiments in chrome stable we can ship the best features possible and build a better browser even when the trade-offs are complex there are three main reasons that applications like chrome need a mechanism like chrome variations first up to enable a new feature using chrome variations to provide control over activation of a new feature is particularly useful for anything that might be more risky in some way or that might affect performance chrome variations can enable the new feature to be rolled out to a subset of users and then chrome engineers can check for differences in performance or look at other types of feedback from the experiment group chrome variations can also be used although this happens more rarely as a safety mechanism to deactivate a feature for example a new networking feature might become subject to a denial of service attack chrome variations can be used to quickly turn off a feature like that since the chrome variations configuration is downloaded every 30 minutes and settings are activated every time you restart chrome by contrast simply updating chrome and waiting for the new version to be propagated to billions of users would be much slower lastly chrome variations can be used to try changes to a feature for example we can try tweaking the number of results shown in the omnibox now chrome variations can also be used to study the long term effects of new features and this is done by using the chrome variations mechanism to hold back features for a small proportion of users you know maybe one percent or less a holdback group can be useful to check for changes and side effects that only become visible over time and this can be particularly important for user interface changes where users are likely to interact with a feature when it's new but you know may behave differently in the long term for example photo picker functionality was launched in chrome but the feature was temporarily held back from a small percentage of users to see how outcomes compared and you know we shipped the new android photo picker to a high proportion of our users and initially saw a significant increase in the number of images that groups shared on the web however in the six months that we had the holdback experiment we saw the lift in usage decrease significantly and you know this was because availability of the new photo picker encouraged sites to add the accept attribute to file input elements yeah which led to a better experience for all users chrome variations is particularly useful for testing one group of users against another for example a feature can be turned on for some users and not others chrome can then compare metrics for each group using what's called telemetry if the setting to help improve chrome's features and performance is enabled then chrome can automatically monitor and send metrics to the chrome backend servers and these metrics might include memory usage page load times or the usage of a browser feature each set of chrome variations tests for a feature is called a study or a field trial and each one has a fixed duration once a study is over for a feature any users who are included in the experimental group get the default chrome setting for the experimental feature you know either enabled or disabled most chrome variations that can be controlled by chrome variations correspond to a flag that can be set from the chrome flags page flags can also be set if you run chrome from the command line and that's what enable features and disable features are for for enterprise customers chrome also provides the chrome variations policy to manage chrome variations experiments now of course we don't recommend disabling chrome variations since that can stop chrome from quickly providing critical security fixes so that's chrome variations to find out more take a look at our article on developer.chrome.com and if you have any questions about chrome variations or you know anything's not clear from this video just contact me via twitter so thanks for watching and be sure to check out the other videos in the chrome concepts series [Music] [Applause] [Music] you