Oil-Drop Walkers - an Analogue for Wave Particle Duality

There has been an ongoing research effort by Yves Couder at CNRS University in France and John Bush an Applied Mathematician at MIT in the USA using fluid mechanics as an analogue to look at the de Broglie-Bohm interpretation of Quantum Mechanics.

It originated by chance when observing the movement of oil droplets on a vibrating water bath.

It is fascinating stuff.

Resources

Phylogeny Again

I have found some really excellent visualizations of Phylogeny—the tree of life—using cladistics or clades.

Mikes Notes

  • Find a way to incorporate all of these into the front end.
  • Compare data sources - are they the same?
  • Do they use API's?
  • Any open-source cladistic tooling?

Upgrading from Java 8 to Java 12

The latest QCon weekly newsletter has a great article on the reasons to upgrade from Java 8 to 12 and the changes to Java support announced by Oracle.

Article key takeaways.

  • Since Java 8 a number of useful new language features have been introduced, along with new tooling, and performance improvements particularly for garbage collection.
  • When choosing to upgrade the choice you face is whether to upgrade to the latest version of Java (12) and be prepared to upgrade every six months; or upgrade to the latest LTS (11) to give yourself up to three years to think about your next upgrade.
  • Don’t be tempted to ignore compiler warnings.  Deprecation is being taken much more seriously in this modern Java world, and both Java 10 and Java 11 removed APIs.  
  • One of the changes from Java 9 was that internal APIs (largely those classes in packages that started with sun.misc.*) were hidden from use. APIs that are not core to the JDK have also been removed in Java 11. These changes may impact your application but there is a clear path to avoid these problems.
  • Once "over the hump" of this first upgrade, it's worth at least testing the application on the latest version of Java every 6 months, for example in CI.

There is a great doc here which discusses OpenJDK etc.

Is Java still Free?

A panel discussion recorded April 9 at QCon London 2019, with video, slides and transcript. The panelists talk about costs, freedom of use, who governs Java/OpenJDK and what providers are for Java infrastructure for the next 5, 10, 15 years.

Mikes Notes

  • The clearest explanations I've heard.
  • Develop on Java 12 OpenJDK.

OneZoom Tree of Life

OneZoom has to be one of the coolest visualisations of the Tree of Life.















It is an interactive map of the evolutionary relationships between over two million species of life on our planet.

There is a great interview with co-creator James Rosindell in the Complexity Explorer newsletter.

Adobe Experience Platform Open API

Adobe has been opening up their platform to integration with open API's.

From the Adobe web site ...


API-First Development

Experience Platform has been designed and built using core principles of API-first development:
  1. The API is the primary interface into Experience Platform.
  2. All Platform functions, including administrative functions, are available via API.
  3. Platform services, Adobe solutions, and 3rd party integrations use the same API.
Guided by these principles, Adobe Experience Platform services and solutions are designed by first looking at the API and ensuring the functionality exists to support developers through low-level CRUD operations. Only once the API is built can the user interface (UI) be constructed by making REST calls to the API. By starting with the API, Experience Platform allows not only Adobe developers but any developer to have the same access to the core logic of Platform services and solutions by means of the same API calls. This principle is not limited to REST-based APIs. It also applies to other interfaces into Experience Platform like message-based, event-driven interfaces that allow developers to build responsive, reactive applications.

Mikes Notes

  • Looks useful
  • Get API documentation
  • JSON/Yamal?
  • Do some tests

The Sound of Hydrogen - iPython

Here is a Jupyter nbviewer of the Sound of Hydrogen which is taken from a blog post on the Jupyter web site. Just experimenting and having fun.




Mikes Notes


Embed Jupyter nbviewer of iJulia on a web page

I'm playing with a Jupyter nbviewer of iJulia programming language. It's a cool way to share a Jupyter Notebook.


Project Jupyter with Brian Granger

Project Jupyter: From Computational Notebooks to Large Scale Data Science with Sensitive Data with Brian Granger. ... from ACM July 2018.

Brian Granger is an Associate Professor of Physics and Data Science at Cal Poly State University in San Luis Obispo, CA. USA.

"Project Jupyter is an open-source project that exists to develop software, open standards, and services for interactive and reproducible computing. The main application developed by the project is the Jupyter Notebook, a web-application that allows users to create documents that combine live code with narrative text, mathematical equations, and visualizations. Since its creation in 2011, the Jupyter Notebook has become a widely-used, open standard for developing, sharing, communicating, and reproducing computational work in scientific computing and data science.

In this talk I will give an overview of Project Jupyter and its open-source software and open standards for interactive and exploratory computing. Examples of its usage across a broad range of industries, disciplines and organizations will be used to illustrate the main ideas upon which Jupyter is founded. I will end by sketching our current work on JupyterLab, JupyterHub, and Binder and show how it is leading to 1) new challenges with large scale data science within complex organizations and 2) legal, ethical and technical questions regarding sensitive data."




Slides (pdf).

Expanding Horizons: Understanding the EHT’s black hole discoveries

From Wikipedia

"The Event Horizon Telescope (EHT) is a project to create a large telescope array consisting of a global network of radio telescopes and combining data from several very-long-baseline interferometry (VLBI) stations around the Earth. The aim is to observe the immediate environment of the supermassive black hole Sagittarius A at the center of the Milky Way, as well as the even larger black hole in the center of the supergiant elliptical galaxy Messier 87, with angular resolution comparable to the black hole's event horizon."






From Perimeter Institute email.


On April 10 at 9 am ET, the Event Horizon Telescope (EHT) collaboration will announce a “groundbreaking result” regarding black holes.

  • Shep Doeleman, EHT Director, Center for Astrophysics, Harvard & Smithsonian.
  • Dan Marrone, Associate Professor of Astronomy, University of Arizona.





Following the announcement, at 11 am ET, black hole experts will explore the implications of the findings during “Expanding Horizons,” a live panel discussion at Perimeter Institute.

On the panel will be:
  • Robert Myers, Perimeter Institute Director and BMO Financial Group Isaac Newton Chair in Theoretical Physics
  • Beatrice Bonga, postdoctoral researcher, Perimeter Institute
  • Asimina Arvanitaki, Perimeter Institute Faculty member and Stavros Niarchos Foundation Aristarchus Chair in Theoretical Physics
  • Brian McNamara, Department Chair of Physics and Astronomy, University of Waterloo



Articles




Mikes Notes

  • Outstanding science in my opinion.

GCP in 4 Words or Less

The Google Cloud Developers Cheat Sheet by the Google Developer Relations Team can be found at GitHub.

"A list of every product in the Google Cloud family described in 4 words or less".

There is a ton of stuff available with direct links to the products, documentation etc.



Google Cloud Next '19

Google is holding a 3-day developer conference Cloud Next '19 in San Fransisco April 9-11. It started today at 4am NZ time.

There are 6 streams running at the same time covering a variety of topics.

All of the conference sessions are available live-streamed and also on-demand. I'm watching all the conference sessions.

Here is a playlist of all sessions.


I'm following;
  • Running on cloud, on-premise and multi-cloud.
  • Data storage.
  • Avoiding vendor lock-in.
  • AI strengths and weaknesses.
  • Security.
  • Product demos.
  • Google Cloud road map.
The conference and many previous conferences are also available on YouTube.

There were 32 recorded sessions from the first day alone, so .... maybe 90 hours of watching coming up.

i18n - Internationalisation and Localisation

Mikes Notes

This project needs to be used by people who speak different languages and write in various ways (left-right, right-left, vertical).

It is possible. However, how well that works with AJAX has yet to be discovered.

Useful information:

Quanta Magazine - Cryptography That Can’t Be Hacked

From an article, written by Kevin Hartnett published April 2, 2019, in Quanta Magazine.

Researchers have just released hacker-proof cryptographic code - programs with the same level of invincibility as a mathematical proof.

Programmers are human, but mathematics is immortal. By making programming more mathematical, a community of computer scientists is hoping to eliminate the coding bugs that can open doors to hackers, spill digital secrets and generally plague modern society.

Now a set of computer scientists has taken a major step toward this goal with the release today of EverCrypt, a set of digital cryptography tools. The researchers were able to prove — in the sense that you can prove the Pythagorean theorem - that their approach to online security is completely invulnerable to the main types of hacking attacks that have felled other programs in the past. “When we say proof, we mean we prove that our code can’t suffer these kinds of attacks,” said Karthik Bhargavan, a computer scientist at Inria in Paris who worked on EverCrypt.

EverCrypt was not written the way most code is written. Ordinarily, a team of programmers creates software that they hope will satisfy certain objectives. Once they finish, they test the code. If it accomplishes the objectives without showing any unwanted behavior, the programmers conclude that the software does what it’s supposed to do.

Yet coding errors often manifest only in extreme “corner cases” - a perfect storm of unlikely events that reveals a critical vulnerability. Many of the most damaging hacking attacks in recent years have exploited just such corner cases.

“It’s some cascading failure, and it’s hard to systematically find because [the events leading to it] are individually all very unlikely,” said Bryan Parno, a computer scientist at Carnegie Mellon University who worked on EverCrypt.

By contrast, Parno and his colleagues have specified exactly what their code is supposed to do and then proved it does that and only that, ruling out the possibility that the code could deviate in unexpected ways under unusual circumstances. The general strategy is called “formal verification.”

“You can reduce the question of how code behaves into a mathematical formula, and then you can check if the formula holds. If it does, you know your code has that property,” said Parno.


PHOTO: Karthik Bhargavan
Karthik Bhargavan, a computer scientist at Inria in Paris, has collaborated on an approach to online security that is provably invulnerable to most types of hacking attacks.

Courtesy of Karthik Bhargavan



Because it’s practically impossible to formally specify the function of complex software such as a web browser, researchers have instead focused on programs that are both critical and amenable to being defined mathematically. EverCrypt is a library of software that handles cryptography, or the encoding and decoding of private information. These cryptographic libraries are innately mathematical. They involve arithmetic with prime numbers and operations on canonical geometric objects like elliptic curves. Defining what cryptographic libraries do in formal terms is not a stretch.

Work on EverCrypt began in 2016 as a part of Project Everest, an initiative led by Microsoft Research. At the time — and still today — cryptographic libraries were a weak point in many software applications. They were slow to run, which dragged down the overall performance of the applications they were a part of, and full of bugs. “I think there’s been some realization from app developers that there’s a disaster waiting to happen,” said Jonathan Protzenko, a computer scientist at Microsoft Research who worked on EverCrypt. “The software world is ripe for something new that does provide [EverCrypt’s] guarantees.”

The main challenge to creating EverCrypt was developing a single programming platform that could express all the different attributes the researchers wanted in a verified cryptographic library. The platform needed the capacity of a traditional software language like C++ and the logical syntax and structure of proof-assistant programs like Isabelle and Coq, which mathematicians have been using for years. No such all-in-one platform existed when the researchers started work on EverCrypt, so they developed one — a programming language called F*.  It put the math and the software on equal footing.

“We unified these things into a single coherent framework so that the distinction between writing programs and doing proofs is really reduced,” said Bhargavan. “You can write software as if you were a software developer, but at same time you can write a proof as if you were a theoretician.”

PHOTO: Bryan Parno
Bryan Parno, a computer scientist at Carnegie Mellon University, uses techniques from mathematics to prove that programs don’t have bugs.

Lauren Demby / Lauren Renee Photography







Their new cryptographic library provides a number of security guarantees. The researchers proved that EverCrypt is free of coding errors, like buffer overruns, that can enable hacking attacks — in effect, provably ruling out susceptibility to all possible corner cases. They also proved that EverCrypt gets the cryptographic math right every time — it never performs the wrong computation.

But the most striking guarantee EverCrypt makes has to do with an entirely different class of security weaknesses. These occur when a bad actor infers the contents of an encrypted message just by observing how a program operates.

For example, an observer might know that an encryption algorithm runs just a little faster when it adds “0” to a value and just a little slower when it adds “1” to a value. By measuring the amount of time an algorithm takes to encrypt a message, an observer could start to figure out whether the binary representation of a message has more 0s or 1s in it — and eventually infer the complete message. “Somewhere deep in your algorithm or the way you implement your algorithm you are leaking information, which can completely defeat the purpose of the entire encryption,” said Bhargavan. Such “side-channel attacks” were behind several of the most notorious hacking attacks in recent years, including the Lucky Thirteen attack. The researchers proved that EverCrypt never leaks information in ways that can be exploited by these types of timing attacks.

Yet while EverCrypt is provably immune to many types of attacks, it does not herald an era of perfectly secure software. Protzenko noted there will always be attacks that no one has thought of before. EverCrypt can’t be proven secure against those, if only for the simple reason that no one knows what they will be.

In addition, even a verified cryptographic library has to work in concert with a host of other software, like an operating system and many common desktop applications, that are typically unverified, and likely will be for the foreseeable future. “We’re not targeting something as complex as a word processor or a Skype client,” said Protzenko, because it’s not obvious how you’d capture in a formal language what they’re supposed to do. “It’s hard to think about the intended behavior of those things.”

Because vulnerabilities in adjacent, unverified programs can undermine a cryptographic library, Project Everest aims to surround EverCrypt with as much verified software as it can. The overarching goal of the initiative is to complete a fully verified implementation of Hypertext Transfer Protocol Secure (HTTPS), the software that secures most web communication. This will involve half a dozen individual software elements like EverCrypt, each of which is formally verified to work on its own and all of which are formally verified to work together.

“Project Everest is trying to build out a larger stack of software that’s all been verified and verified to work together. Over time we’re hoping the frontier [of verified software] will continue to grow,” said Parno."

Mike's Notes

  • Follow up on Project Everest (Microsoft Research) 
  • Look for ongoing test results

Chris Richardson - Documenting a service using the microservice canvas

This is a copy of an excellent blog article by Chris Richardson from his web site. He also has more information about this on GitHub.

In What’s a service - part 1? I described the key aspects of a service, which includes its API, and its dependencies. A good way to document a service and its structure is to use a microservice canvas. A microservice canvas is concise description of a service. It’s similar to a CRC card that’s sometime used in object-oriented design.

I first read about the canvas in a 2017 DZone article by Matt McLarty and Irakli Nadareishvili. I’ve since adapted its structure so that it emphasizes the interface (top of the canvas) and the dependencies (bottom of the canvas) and de-emphasizes implementation (middle of the canvas).

Here is an example of a microservice canvas. It describes the Order Service, which is part of my book’s example FTGO application.

A service’s external view

The service’s external view is described by the following sections:
  • name - name of the service
  • description - a brief description of the service
  • capabilities - the business capabilities implemented by the service
  • service API - the operations implemented by the service and the domain events published by the service
  • quality attributes - the service’s quality attributes, which are also known as non-functional attributes
  • observability - includes health check endpoint, key metrics, etc.

A service’s dependencies

A service’s dependencies are described by the dependencies section, which consists of two parts.
  • invokes - the operations which are implemented by other services that this service invokes
  • subscribes - the messages, which include events, that this service subscribes to

A service’s implementation

The canvas can also describe the service’s implementation, such as its domain model.

Example canvas

To learn more

Mike's Notes

  • Use this template, which is available on GitHub.
  • Credit both Chris & Matt.
  • It closely matches what I use anyway, but it is much better laid out.
  • Most of these metadata fields are already completed.
  • The existing project documentation generator could quickly work with this template to batch output HTML and PDF files.
  • Will not be using Saga's, however, can work with workflow messaging.

InfoQ Weekly Architects Newsletter

InfoQ must be the best web site I have come across in many years for broad vendor and language-neutral coverage on
  • Software development
  • News
  • Videos
  • Books
  • Articles
  • Videos with transcripts
  • Podcasts
  • Free guides (PDF)
There is also great conference coverage of QCon Conferences with all speakers presentations available. The presentations have a lot of depth.

It is well worth subscribing to. I get a weekly Architects Newsletter which is a concise digest of the topics I'm following, and there is no hard sell.

Every week I get links to download free O'Reilly Media software books.

This resource has probably been the key ingredient over the last six months for expanding my horizons on what it is possible to build today.

DevOps

DevOps is a set of software development practices that combines software development (Dev) and information technology operations (Ops) to shorten the systems development life cycle while delivering features, fixes, and updates frequently in close alignment with business objectives. [1]

From Wikipedia

Agile software development has broken down some of the silos between requirements analysis, testing and development. Deployment, operations and maintenance are other activities which have suffered a similar separation from the rest of the software development process. The DevOps movement is aimed at removing these silos and encouraging collaboration between development and operations.

DevOps has become possible largely due to a combination of new operations tools and established agile engineering practices, but these are not enough to realize the benefits of DevOps.

Even with the best tools, DevOps is just another buzzword if you don't have the right culture.

The primary characteristic of DevOps culture is increased collaboration between the roles of development and operations. There are some important cultural shifts, within teams and at an organizational level, that support this collaboration.



By Rouan Wilsenach
From MartinFowler.com


Multi-tenancy

From Wikipedia

The term "software multi-tenancy" refers to a software architecture in which a single instance of software runs on a server and serves multiple tenants. Systems designed in such a manner are often called shared (in contrast to dedicated or isolated). A tenant is a group of users who share common access with specific privileges to the software instance. With a multi-tenant architecture, a software application is designed to provide every tenant with a dedicated share of the instance - including its data, configuration, user management, tenant individual functionality and non-functional properties. Multi-tenancy contrasts with multi-instance architectures, where separate software instances operate on behalf of different tenants.

Mikes Notes

Pipi 6 was built to support multi-tenancy.
  • Deployments can be sole-tenancy or multi-tenancy
  • Uses database sharding

Software as a Service

Software as a service (SaaS /sæs/[1]) is a software licensing and delivery model in which software is licensed on a subscription basis and is centrally hosted. It is sometimes referred to as "on-demand software", and was formerly referred to as "software plus services" by Microsoft.[2] SaaS is typically accessed by users using a thin client, e.g. via a web browser. SaaS has become a common delivery model for many business applications, including office softwaremessaging software, payroll processing software, DBMS software, management software, CAD software, development software, gamificationvirtualization[3] accountingcollaborationcustomer relationship management (CRM), Management Information Systems (MIS), enterprise resource planning (ERP), invoicing, human resource management (HRM), talent acquisitionlearning management systemscontent management (CM), Geographic Information Systems (GIS), and service desk management.[4] SaaS has been incorporated into the strategy of nearly all leading enterprise software companies.[5][6]

From Wikipedia

Ontology

In computer science and information science, an ontology encompasses a representation, formal naming, and definition of the categoriesproperties, and relations between the conceptsdata, and entities that substantiate one, many, or all domains.

Every field creates ontologies to limit complexity and organize information into data and knowledge. As new ontologies are made, their use hopefully improves problem solving within that domain. Translating research papers within every field is a problem made easier when experts from different countries maintain a controlled vocabulary of jargon between each of their languages.[1] 

Since Google started an initiative called Knowledge Graph, a substantial amount of research has gone on using the phrase knowledge graph as a generalized term. Although there is no clear definition for the term knowledge graph, it is sometimes used as synonym for ontology.[2] One common interpretation is that a knowledge graph represents a collection of interlinked descriptions of entities – real-world objects, events, situations or abstract concepts.[3] Unlike ontologies, knowledge graphs, such as Google's Knowledge Graph, often contain large volumes of factual information with less formal semantics. In some contexts, the term knowledge graph is used to refer to any knowledge base that is represented as a graph

From Wikipedia