Open Source Landscape v2 including more categories

Wow, this turned out to be more involved especially in the CMS area. So much human capital and VC funding went into this. Great that we can all leverage it.

The laws of software

There a a couple of laws the govern software. Let's review some of the most common, starting with my favorite coming from a big company.

Conway's Law
Basically this states that software architecture would mirror organizational boundaries. Not surprisingly so since teams optimize to reduce cross team dependencies. It can however be put to good use. If you want to ensure good boundaries in your architecture, mirror those in your organization. Anecdotally I have seen when client and server are in one team. Amazing how the API's become tailored for a specific client and generality goes out the door.

Moore's Law
Probably the most common, this law has to do with the projected growth in computing. Moore's law states that the number of transistors in CPU's would double every 2 years. This made scaling up very tempting and bought many system and engineers some time. Just add ram and bigger CPU's. Scaling out, FWIW is a much better approach but does require some re-architecting if not done from the get go.

Shannon's Law
Shannon's law covers the speed at which data can be transmitted over a specific medium keeping in mind the signal to noise rate. Hard to imagine this law having an impact in the era of 10 Gig-E networks, but does impact wireless and other transmission protocols.

Little's Law
Little's law has application in designing concurrent systems and the relationship between latency and throughput whether a message queue, thread pool or web server. Simple states that the concurrent processing is a function of the incoming transaction rate multiplied with the processing time for every transaction. Example, assuming a consistent web page visit, if you page loads times go up, the CPU usage goes up, which can increase the page load time...you get the idea.

Open Source Technology Landscape

Having benefited immensely from LUMA partners version of the advertising landscape, I decided to create one to map out the open source technologies/projects. Not intended to be comprehensive, but do let me know if I missed key projects. Please opine.

Architecture Tenets

Over my career working in the online / distributed software arena, I have honed in on the following architecture tenets that I tried to deploy in any internet scale architecture problem.

Separation of Concerns
Build autonomous, decoupled services - SOA

• Define Clear contracts (Semantic, SLA)
• Define Explicit boundaries
• Backward & Forward Compatibility
• Versioning

Design to Scale Out
Don’t rely purely on Moore’s law. Systems should be decoupled and/or partitioned to scale out

• No Single Choke points
• Partitioning should be Fine grained
• Be able to run on Commodity Hardware
• Designed for GEO readiness

Design for Change
Changes are the norm not the exception

• Design applications to be modular and extensible
• Design components to be flexible and configurable

Simplicity
Complicated designs fail in complicated and expensive ways

• Simplified Operability (Instrumentation/Monitoring/Tools)
• Simplified Debugging (Root Cause Analysis) 
• Simplified Ramp up 
• Diagnostic Capabilities

Design for Failure
Failures are the norm not the exception

• No Single Point of Failure 
• Services should be resilient (Run in the wild) 
• Graceful Degradation of services 
• Deployments should be automatable 
• Ability to roll back releases