What’s the difference of creation, innovation, and improvement? Creation is making something, like making a pot or a baby. Innovation is making something different and new, like making an AI. Improvement is making things better, it does not have to be new. Different and new is relatively easy. Doing something that’s genuinely better is very hard. - Jonathan Ive Why Innovation matters to business Companies want to make more money for a longer time. Predicting the future is very profitable. The best the way to predict the future is to create it. - Peter Drucker How to do innovation In Two steps: exploration and exploitation. ...

Data description: Daily new confirmed covid cases of every province in the China mainland area. Move the slider to see daily data since Feburary 2020. Click on the button to travel through time. The source: National Health Commitee Data are updated weekly. demo

Case 1. “/whatis”: a slack command server Business: When Slack users input a command like “/whatis GDPR”, search on wiki pages and respond in 3 seconds what GDPR is. Generation 1. servers on machines ElasticSearch: to respond in less than a second Ngrok: to pierce the private network Gen 2. containerized Docker: to make deployments reliable Gen 3. Cloud (AWS EC2 + Amazon ElasticSearch Service) To get rid of on-premise server room, which occasionally suffers from power outages. ...

A brief history The beginning: trade software for money Distribution problems: installation, licensing, upgrades Client / Server Distributed services (Conway’s law) Exploiting resources: Andy and Bill’s law Outsourcing hardwares Outsourcing Infra-Softwares Cloud industry Software delivery concept evolution (SWOT analysis) Concept strength weakness opportunity threat library extension and consolidation requires build; cannot deploy independently semantic versions dependency hell; backward / forward compatibility runtime binary + scripts deploy independently; cross-platform; slow; cannot conceal information; need a large runtime to do small things fast development hard to maintain; cannot hide sensitive information binary packages / installers does not require build; has more complexity cannot cross-platform; cannot fully control the deployment environment sell as products, license and fee hard to upgrade; security risks. services on servers more control on the deployment sites; upgrade anytime performance / experience depends on network connections APIs; social networks; zero down time; freeminum network security; do not control client upgrades distributed / tiered services independent deployment and evolution, divide and conquer coupled with the human structure (Conway’s law) business can hire more people single point of failure; service explosion and governance; inconsistency virtualization higher efficiency on hardware resources start / stop is slow (in minutes) scale up and down; pay as you go more security loopholes containerization a new interface to deliver and run software fast and cheap docker / containerd dependency; loss of observability and performance; docker image repos; CI / CD docker vendor turf wars container orchestration container operations still need to maintain nodes multi-tenancy container clusters; sidecars vendor lock-in on k8s designs and cloud providers; too many yamls; loss of dev agility service mesh observability, network routing, more ways of load balancing and flow control more network hops and latencies; over design for most systems; chaos engineering, system-level performance testing Sidecar errors may be a single point of failure Serverless divide between functional and non-functional requirements reinventing PaaS; maybe a wrong division; hard to work on special non-functional requirements. low-code, no-code: faster development with less qualified developer. too many hidden layers; vendor lock-in IDC outsourcing hardwares performance dependent on IDC connections and power supply multi-region availability; CDN; pay as you go on hardware layer. data security; cannot evolve on hardware / software integrations Cloud outsourcing softwares and system operations system has more hidden layers easy infrastructure; pay as you go on more layers; no need to maintain hardwares vendor lock-in; Hotel California pricing

why? the only proper way to do any kind of research for quality assurance goals find and fix bugs as early as possible ensure code changes work as expected without breaking any released features Tests as in software system development name when to do done by whom granularity whitebox/blackbox tools business impact unit test during coding engineers per function whitebox code libs none smoke test after coding engineers per api, per service blackbox cucumber, etc. none integration test system integration engineers per system blackbox cucumber, etc. none performance test before release engineers per system blackbox gatlin, jmeters, etc. none regression test during & after release engineers, product owners per system blackbox cucumber, etc. low alpha & beta test after release product owners, selected users per product blackbox high Test case design criteria : what factors contribute to a “good” test case design (and how good is enough)? collectively exhaustive: cover all lines of codes / all possible use cases mutually exclusive: use as few test cases / codes as possible rational methodology: repeatable: given inputs should have stable outputs explicit inputs and outputs without side effects ( “pure” ) eliminate implicit variables, e.g. testing person biases, environment assumptions and dependencies change only one variable per case Test framework criteria adaptive to code, input and environment changes open for extension, closed for modification fully integrated in all phases of development processes Tests are basically simulations Why test on simulations instead of “real” ones? test on real subjects are costly, in some cases with business impacts separation of testing subjects ( by defining a test scope ) to eliminate implicit variables and side effects Prerequisites for doing simulation tests subject codes rely on interfaces instead of specific implementations subject codes use dependency injection (all dependencies are specified in object creation) aspect oriented programming: subject codes have middleware injection points for testing purposes Variances of simulation objects excerpt from http://xunitpatterns.com/Test%20Double%20Patterns.html ...