Many Java developers have been practicing continuous integration and some form of continuous delivery for the past decade. Innovative books like “Java Power Tools” by John Smart provided the guidelines and frameworks to make this possible. Technologies have obviously changed within the last ten years, and so have associated programming and architectural styles. In particular, business teams within organisations have increasingly demanded that IT teams become more flexible and be capable of rapidly responding to change in customer preferences and market conditions.
The emergence of dynamic and programmable compute resources and deployment platforms, combined with teams and organisations exposing Application Programming Interfaces (APIs) as products has resulted in the architectures that Java developers are creating to converge towards component/service/function-based architectures. All of these factors have led to (and in turn, have been driven by) the emergence of popular movements such as Agile, Lean, DevOps, Cloud Computing, Programmable Infrastructure, microservices, and serverless or Function-as-as-Service (FaaS).
The Changing Role of a Developer and Architect: Craftspersonship
Since the 2010’s began, the role of a “programmer”, “developer” and “architect” has arguably changed to include more responsibilities. You’ll learn more about this in my blog, but if you are keen to dive even deeper into these concepts and how they relate to your career development, professionalism and becoming a “craftsperson”, then we recommend the following books:
- The Software Craftsman: Professionalism, Pragmatism, Pride, by Sandro Mancuso Mancuso
- The Clean Coder: A Code of Conduct for Professional Programmers, by Robert C. Martin
- The Passionate Programmer: Creating a Remarkable Career in Software Development, by Chad Fowler
Need for Business Speed and Stability
During his time as Cloud Architect at Netflix, Adrian Cockcroft talked a lot about about ‘time to market’ being a competitive advantage, and in many modern markets ‘speed kills’. Uwe Friedrichsen, CTO at codecentric, has also talked extensively about this trend beginning in the 1980s: globalisation, market saturation and the Internet led to highly competitive and dynamic ecosystems. The markets became highly demand-driven and the new biggest challenge of the companies was to adapt to the changing demands of the customers quickly enough. The key driver changed from “cost-efficient scaling” to “responsiveness”.
Over the same time period, the move to public commodity infrastructure (“Cloud”) in combination with increasing transaction value flowing through global computer systems has meant that new failure modes are being discovered, and new attackers are emerging from the shadows. This has caused the need to balance stability and security against the requirement for speed. Often this isn’t an easy balance to maintain.
Continuous delivery is achieved when stability and speed can satisfy business demand.
Discontinuous delivery occurs when stability and speed are insufficient.
- Steve Smith (@AgileSteveSmith)
Accordingly, you now need to create applications that support rapid, safe and stable change, and continually ensure that you are meeting these requirements through automated testing and validation.
Rise of the API Economy
Application Programming Interfaces (APIs) are at the core of the Internet and a modern developer’s daily life - RESTful services are the de facto way to expose and consume third-party online business services. However, as stated by Jennifer Riggins when attending the 2017 APIDays conference, what people might not realise is how much the API will be at the center of the future technology, and part of every connected person’s daily life. APIs will continue to play a central role in trends like chatbots and virtual assistants, the Internet of Things (IoT), mobile services, and so much more.
APIs are also being increasingly consumed as “shadow IT” by departments that were traditionally less “tech-savvy” like marketing, sales, finance, and human resources. “Mediated APIs” - APIs that act as bridges between new and old applications - are becoming increasingly popular, as they provide adaptations and opportunities for innovation in businesses that have considerable investment locked within legacy infrastructure. Gartner, the US-based research and advisory firm, suggests that concepts such as the “API marketplace” and the “API economy” are becoming increasingly important within the global economy.
As the API marketplace becomes more sophisticated and widespread, the risks for failure and security issues become more apparent. APIs have made technology more accessible than ever, which means that Enterprise Architects, the traditional bastions of technology adoption, are no longer the gatekeepers for technical decision-making. Accordingly, this empowers every developer in an organisation to innovate, but at the same time can lead to unintended consequences. It is essential to codify not only functional requirements for an API - for example using Behaviour-Driven Design (BDD) and automated testing - but also non functional (or cross-functional) requirements and Service Level Agreements (SLAs) related to security, performance and expected cost. These must be continually tested and validated, as this has a direct impact in the product being offered to customers.
Opportunities and Costs of the Cloud
It can be argued that the cloud computing revolution began when Amazon Web Services (AWS) was officially launched in March 2006. Now the Cloud Computing market generates $200+ billion in revenue annually. Cloud computing technologies have brought many advantages -- on-demand hardware, rapid scalability and provisioning, and flexible pricing -- but have also provided many challenges for developers and architects. These include the requirements to design for the ephemeral nature of cloud computing resources, the need to understand the underlying characteristics of a cloud system (including ‘mechanical sympathy’ and fault tolerance), and the requirement for an increase in operational and sysadmin knowledge (such as operating systems, configuration management, and networking).
Developers unfamiliar with cloud technologies must be able to experiment and implement continuous testing with these deployment fabrics and platforms, and this must be done in a repeatable and reliable way. Early testing within a build pipeline using application deployed on infrastructure and platforms that are as like production as possible is essential to ensure assumptions on performance, fault tolerance and security are valid.
Modularity Redux: Embracing Small Services
The combination of the need for speed from the business, the adoption of REST-like APIs, and the emergence of cloud computing has provided new opportunities and challenges to software architecture. Core topics in this space include the scaling both the organisation aspects of developing software (e.g. Conway’s Law) and the technical aspects (e.g. modularisation), and also the requirement to deploy and operate parts of the codebase independently of each other. Much of this has been incorporated within the emerging architectural pattern known as the “microservices”.
My blog will discuss the drivers and core concepts of microservices in and explore how this helps and hinders the implementation of CD. A further introduction to microservices can be found in Christian Posta’s Microservices for Java Developers (O’Reilly), and a more thorough treatment can be found in Sam Newman’s Building Microservices (O’Reilly) and Amundsen et al’s Microservice Architecture (O’Reilly). At a high-level the building of Java-based microservices impacts the implementation of CD in several ways:
- Multiple build pipelines (or branches within a single pipeline) must be created and managed.
- Deployment of multiple services to an environment now have to be orchestrated, managed, and tracked.
- Component testing may now have to mock, stub, or virtualize dependent services.
- End-to-end testing must now orchestrate multiple services (and associated state) before and after executing tests.
- Process must be implemented to manage service version control (e.g., the enforcement of only allowing the deployment of compatible interdependent services).
- Monitoring, metrics, and application performance management (APM) tooling must be adapted to handle multiple services.
Decomposing an existing monolithic application, or creating a new application that provides functionality through a composite of microservices is a non-trivial task. Techniques such as context mapping, from domain-driven design, can help developers (working alongside stakeholders and the QA team) understand how application/business functionality should be composed as a series of bounded contexts or focused services. Regardless of how applications are composed, it is still vitally important that both individual components and the system as a whole is continually being integrated and validated. The need for continuous delivery only increases as more and more components are combined, as it becomes near-impossible to manually reason about their combined interactions and functionality.
Impact on Continuous Delivery
Hopefully this exploration of the requirements of modern Java applications has highlighted the benefits -- and some cases the essential need -- of continuous delivery to ensure that software systems provide the required functionality. The changing requirements, infrastructure and architectural styles are just parts of the puzzle though, and at the same time new platforms have emerged that have either codified several of the architectural best-practices or have attempted to help address some of the same problems.
