# Why? * I realised the following some weeks ago: I now maintain deployment tooling for one of the most important websites in the world. When I make a mistake, and Wikipedia goes down, the whole world will notice. * How can I avoid making at least some mistakes? * How can I avoid the New York Times and Hacker News discussing my shortcomings? ----------------------------------------------------------------------------- # We have no safe place to... * Try changes to train tooling * Try out train tooling to see if it still works * Learn how to conduct the train * Experiment with changes to how we do the train ----------------------------------------------------------------------------- # Theses * Changing `scap` or `deploy-promote` is plain old software development * Changing how we do the train is very similar to software development * We should treat the train as a software development project ----------------------------------------------------------------------------- # (Controversial?) opinion on development (1/9) * Agile is not wrong * Agile is not right * Same for every other formal method or methodology * Higher stakes require more formalism ----------------------------------------------------------------------------- # (Controversial?) opinion on development (2/9) * Have a rough idea of the end goal - this will change and become clearer the closer you get - that's OK, part of the process is figuring out what you (or your users or customers or stakeholders) really, really want - don't obsess about getting this exactly right in the beginning ----------------------------------------------------------------------------- # (Controversial?) opinion on software development (3/9) * Make something that sort of works to start with - a prototype, spike, wireframe, sketch, whatever - it can be limited, bad, ugly, and wrong - a project with a million commits starts with hello, world ----------------------------------------------------------------------------- # (Controversial?) opinion on software development (4/9) * Iterate quickly, more or less towards the end goal - have a very clear goal for each iteration - get feedback at the end of the iteration, to feed into forming goals for future iterations - a week is usually enough for one iteration - many weeks is too long: too much changes in the world in that time - it's better to spend a week going in the wrong direction than a month ----------------------------------------------------------------------------- # (Controversial?) opinion on software development (5/9) * Experiment, make mistakes, learn - "what happens if I press this button?" - if nothing bad can happen, just press the button, and then you'll know - make sure it's safe to press any button ----------------------------------------------------------------------------- # (Controversial?) opinion on software development (6/9) * If something is painful, do it more often ----------------------------------------------------------------------------- # (Controversial?) opinion on software development (7/9) * Smooth away unnecessary friction ----------------------------------------------------------------------------- # (Controversial?) opinion on software development (8/9) * Acceptance criteria are the most important part of a software project - knowing what to do is harder than knowing how to do it - automated tests are more important that production code ----------------------------------------------------------------------------- # (Controversial?) opinion on software development (9/9) * It's not finished, until it's run repeatedly - can your web server handle one billion sequential trivial read-only requests, however slowly? - can you conduct the train every week for a year? ----------------------------------------------------------------------------- # On development speed * Edit, build, run, test, debug - the inner loop of software development - if the loop is slow, development is slow: changes take a long time to make * Anything that slows down the inner loop is bad - if the whole world notices when you make a mistake, you are careful making changes - the careful developer is a slow developer * Software development always involves making mistakes - developing things quickly requires making mistakes fast - each mistake teaches you something - but mistakes should be cheap, safe, harmless ----------------------------------------------------------------------------- # train-dev * A safe place to make mistakes related to development tooling * Simulates the production environment sufficiently that if things work in train-dev, they hopefully work in production * Does not try to be an exact replica of production * Will not always be "right", but "sometimes good enough" is vastly better than "try it in production and take down Wikipedia if you're wrong" * We'll make it be closer to production over time * Iterate, fix discrepancies as we find them ----------------------------------------------------------------------------- # Overview * Nested virtual machines * Outer VM provides an environment in which inner VMs operate in isolation * Inner VMs provide the various servers and services needed to conduct the train - git server ("Gerrit") - a deploy server ("deploy1001.eqiad.wmnet") * The goal is to be able to run all the steps of the train inside the train-dev environment ----------------------------------------------------------------------------- # Current status * The first step works * `scap prep 1.35.0-wmf.34` * Real scap, not one modified for train-dev * Does not access the Internet outside train-dev * An incomplete, bad, ugly, and wrong first step * Let's start iterating? ----------------------------------------------------------------------------- # Do please try this at home! * train-dev repository on Gerrit * `vdc/README.md` has instructions * * There are pre-built images that you can try, no need to build your own. But you need to give Lars your SSH public key first so the next build of the image will give you access the VMs. ----------------------------------------------------------------------------- # Legalese Copyright 2020 Wikimedia Foundation This content is licensed under the Creative Commons Attribution-ShareAlike 4.0 International ([CC BY-SA 4.0][]) licence. [CC BY-SA 4.0]: https://creativecommons.org/licenses/by-sa/4.0/ --- title: "train-dev" subtitle: "The light in the tunnel should not be a locomotive" author: "Lars Wirzenius / Wikimedia Foundation" date: "Version for 2020-08-03 Lunch and learn" ...