Daniël Jumelet, zaterdag 19 november 2011
New approach needed for shaping infrastructure landscapes in a Cloud era
Can architecture really be applied to infrastructure? This is certainly possible thanks to a new, functional approach to the infrastructure formation process. This approach enables architects to fulfil their responsibilities better within this field of expertise, while at the same time providing them with a workflow that is practical.
For many years, infrastructure and architecture seem to have been at odds as a rather unfortunate combination. Leading IT architects have long maintained that infrastructure is really not an architectural subject, but that it belongs purely to the engineering domain. When reviewing many architectural methods with criticism, we can roughly distinguish – as far as infrastructure is concerned – two possible approaches, neither of which work very well: infrastructure is either approached in a superficial and global manner (infrastructure as an application platform or - container, network, storage), or a highly technical approach can be observed, with impacts at the level of structures and topographies (such as stacks and network diagrams). The first approach does not work, because it’s capacity for expression is too small; infrastructure remains a vague and abstract concept, and the development of an infrastructure landscape cannot be shaped and controlled. The second approach is, in fact, engineering, because it focuses on the technical construction, i.e. on ‘how it works’. Because of the high degree of detail and the jargon that is used, however, this type of structure is not accessible to other architects and disciplines.
TOGAF®v9 [The Open Group, 2009] is a good example of this situation, because it is shows both approaches: on the one hand, the infrastructure is scaled down to ‘Platform’ in the Content Metamodel, while on the other hand, a huge stack of technical components is brought together under the Technical Reference Model, which is then given a somewhat functional flavour by the addition of the word ‘service’. There’s no doubt that TOGAF®9 is very useful on various fronts, but not as a tool to model and develop infrastructure. This leads to the inevitable question: which approach to infrastructure architecture does work? The answer to this question is provided by one of the men behind the development of the first 8-bit computer (the IBM System/360), Gerrit A. Blaauw.
Blaauw was not only responsible for the development of the revolutionary System/360 computer; he also documented the design process that he and his colleagues followed at the same time. He defined a workflow for the development of automated systems that was already quite manageable in 1972(!). Blaauw distinguishes between three abstraction levels corresponding to three design phases: Architecture, Implementation and Realisation [Blaauw, 1972].
Architecture is the phase during which the required functionality is defined; the functions are thus defined in a manner that is completely neutral with regard to technology. This phase uses archetypes and basic structures to model and determine the functionality.
In the Implementation Phase, the architecture is translated into a model of the specific manner in which this functionality will be applied, thereby defining principles and guidelines for the technical designer.
Finally, during the Realisation Phase, the tools with which the solution will be materialised are determined, and a technical design is prepared.
A good example that can be used to clarify Blaauw’s three design phases is the analogue clock. The architectureof the analogue clock (the basic structure) can be described as ‘indicating time’ using a big hand, a small hand and a clock face. All analogue clocks conform to this structure. This means that, once a child has learned to tell time, it can do so anywhere, irrespective of the nature and design of the clock. In other words: the child knows/recognizes the architecture of the analogue clock.
If the principle of the analogue clock is used for a specific purpose, the implementation attributes enter into the picture. The specific purpose requires the clock to be designed in a specific way. Let us consider the clocks in a railway station. An important feature is that all clocks within the railway station and on every platform must indicate the same time. If they would not, this would cause confusion and uncertainty amongst the travellers: “Can I still catch my train, and which one of the clocks can I trust?” The architect must make a choice between the different ways in which multiple clocks can be synchronized. He could, for example, opt for the application of highly secure movements, with a deviation of a few seconds per month at the most. In case of a small number of clocks, an isolated context could perhaps be an excellent choice. However, when dealing with large numbers of clocks in many locations, it is very likely that it is better to install a network between simple clocks, with which the clocks can be synchronized every minute. In the end, this was the solution selected by Dutch Railways, and it can be identified by the second hand, which ‘waits’ briefly every minute before it continues. Please note: the guidelines are also still technology neutral at this stage, and no technical components have been determined so far. It is, of course, possible to refer to technical components and/or protocols in the guidelines, but the technology itself is not included at this level. This is the subject of the next stage.
In the realization phase, the guidelines developed by the architect are applied to a technical design, in which the construction of the solution is developed. This is therefore the blueprint of the railway station clock, which is used by the constructor to build one or a whole series of physically identical copies of this clock.
The stages defined by Blaauw can easily be applied to the creation and adaptation of infrastructure landscapes. What is required here is a vocabulary that can be used to express the functions used in the infrastructure landscape. The standard infrastructure jargon is not adequate, because it is too technical. On the basis of the DYA|Infrastructure methodology [Jumelet, 2009], and the community that has gathered around it, a new set of definitions has been developed for this reason. Based on Blaauw's concepts, four different types of definitions can be distinguished, which express the infrastructure functionality:
Single functions at the architecture level (Building Block Types)
Multiple functions at the architecture level (Pattern Types)
Single functions at the implementation level (Building Block Variants)
Multiple functions at the implementation level (Pattern Variants)
To the architect, a Building Block Type has the character of an atomic or indivisible universal infrastructure function. One or more Building Block Variants can be derived from a Building Block Type, and are the instantiation of this type for a specific purpose. An example is the universal ‘Message Engine’ function type, a specific variant of which can be defined that is suitable for the processing of application messages, as well as another specific variant that describes the processing of user mail.
Two or more Building Block Types together form a Pattern Type, which describes a model for an infrastructure solution in a universal manner that is neutral with regard to technology and organisation. An example of this is ‘Message Handling’, which bundles together a number of functions to form a model for a total solution that processes messages. In addition to ‘Message Engine’, this Pattern Type also includes the ‘Message Store’, ‘Message Distribution’ and ‘Message Filtering’ functions.
Pattern Variants are derived from one or more Pattern Types. These describe solutions for a specific purpose, in which the Pattern Types act as basic structures. In this case, the Pattern Variants consist of two or more Building Block variants. As a model, this can be represented as follows:
The complete information model (including quality attributes and other definitions that play a role in the modelling) can be found in the section 'Decomposition and Modeling' of the DYA Infrastructure Repository, the infrastructure architecture community knowledge base, sponsored by Sogeti [Sogeti, 2010].
It turns out that the development of a new vocabulary is rather a complex task. Up to date, no solid rules have been established, and it is a matter of carefully deciding, based on what works best by testing the vocabulary in practice. To stay with the example of the clock: Is it useful to consider the short and the long hand as two separate functions (where one function indicates the hours and the other the minutes)? Or are they two variants of the same type, i.e. the ‘hand’ function? Another function situated more in the infrastructure domain: Is ‘Authentication’ a function, or is it a process that is supported by a combination of the ‘Identity Validation’ and ‘Identity Store’ infrastructure functions? Several possibilities are usually valid, but the choice is defined by a particular idea about modelling around the function(s).
From its early beginnings, the community around DYA|Infrastructure has been involved in the development of the new vocabulary. The definitions at the level that Blaauw called architecture (Building Block Types and Pattern Types) are so archetypal that they can, in fact, be used for any organisation without any modification. The establishment of these definitions therefore leads to a reference architecture that can be used directly by any infrastructure architect.
A knowledge-based system based on a semantic wiki was selected for the registration of the definitions: the DYA|Infrastructure Repository (DIR). The universally valid components of this repository have been made available on the Internet without limitations under a flexible license (Creative Commons). There have been significant effects of the availability of (the content of) this repository, where the strong expansion of the community of infrastructure architects is one of the most appealing. The repository provides a low threshold entry level for this profession, based on a functional approach, which has become available for many infrastructure architects. However, this does not mean that the new vocabulary is now finalised. It is a language that will continue to be fine-tuned and developed further within the community.
With the establishment of the new vocabulary, it has become possible to work through Blaauws infrastructure design steps in a structured and efficient manner. The Pattern Types are used as a basis on which we can define the Pattern Variants that are used to model infrastructure solutions for a specific application. How does this work? Below, an example of the creation of an architectonic description of a Cloud hosting service that is used to host internal applications with Best Effort availability is presented.
The first stepin the design process is to select one or more Pattern Types to form the basic structure of the final solution. These Pattern Types will serve as a basis for the model that is to be developed in the next step, the Pattern Variant.
Figure 2: Architecture model of the Application Hosting solution (Pattern Type)
As the Pattern Type forms a basis, the architect must determine whether or not each of the optional components (shown as shaded) will be included in the solution. Other Pattern Types or independent Building Block Types may also be added. In our Cloud Hosting service example, the architect decides that the Scheduler and Transaction Coordination functions will not be needed, nor will the final solution require the use of (a variant of) Data Storage, because Data Management is sufficient. In summary, it can be stated that the conceptsthat will apply to the design of the required solution are selected in this step.
The second step is the identification of a Pattern variant, which is based on the above Pattern Type in this example. A Pattern variant is outlined here, taking the following functions, requirements and features into account:
This Pattern variant describes the implementation guidelines for a solution supplied by application hosting, with a Best Effort availability and making use of a Cloud service.
This solution is intended to be used for experimental and/or short-term business application hosting requirements. It is intended for internal access/internal use, and must be made available to end users by means of a Cloud Portal, which serves as an interface for Cloud-based applications. As this solution is intended for very fast delivery, no changes can be made to the standard solution.
Features and instructions for use
Very fast delivery for the hosting of experimental and/or short-term business applications
Only internal access to the hosted applications permitted
Only best-effort availability guaranteed
Makes use of Cloud Providers’ standards; it is not possible to add additional components
If necessary, Cloud Integration solutions that are already available must be used:
Data Management - Cloud Internal Usage
Asynchronous Messaging - ESB
Workflow Orchestration - Common Cloud Application Direction
Makes use of internal Authentication & Authorisation services; the use of additional provisions for Identity & Permission Validation is not permitted.
Figure 3: Example of an implementation model for the Application Hosting solution, focused on an application realised via the Cloud (Pattern Variant).
Note that for each infrastructure function that was selected, a specific variant is designated. These function variants can be solution-specific, but it is also possible that these variants are shared by a (large) number of applications. In addition, the infrastructure facility is linked to adjacent (already existing or still to be realised) infrastructure facilities. At this point in the architecture process, the architect can also determine the required infrastructure-specific quality aspects of the solution to be achieved (such as availability, accountability and manageability), and link and translate the relevant architecture principles of the organisation. In summary, it can be stated that the requirements for the specific solution are determined in this step on the basis of the concepts that were selected in the previous step. These requirements are then translated into guidelinesfor the design.
In the example given, functions like Control Interface.Cloud Platforms and Presentation Aggregation.Cloud Portal-Internal Use are characteristic illustrations of functions of which an architect wants to promote shared usage in order to standardize and consolidate the infrastructure landscape. Shared use of a Control Interface improves manageability, shared use of a common Portal helps the accessibility and usability of (Cloud) applications by end-users. The modelling exercise makes it feasible to explore the currently achievable options for strengthening the integration and unification of the infrastructure landscape, making it more economical to utilize while delivering better functionality and performance to its users. The reason for this is that the Pattern Types serve as structures for guiding the important design decisions the architect has to make.
The third step is the creation of a Design Outline. The Design Outline functions as an artist impression or, in other words, a design study. It represents (schematically) how the architect aims to realise the facility. The form in which the Design Outline is presented depends on the architects target group and the resources that are available to him. It is therefore quite possible to also outline the construction with ArchiMate® in a more or less abstract manner, because ArchiMate® contains the language elements required for depicting infrastructure components. It is also possible to generate network diagrams and/or drawings with system symbols (as used by Microsoft, for example). In some cases, it may be good practice to describe the solution in everyday language. A possible interpretation of the above model, using ArchiMate®, is shown below (with a highly simplified view):
Figure 4: Example of a Design Outline, outlining the global (partial) development of the implementation model shown above (Pattern Variant).
The Design Outline does not supply the design itself, but contains specific instructions for the design using the terminology (concepts and technical specifications) that will also come up in the design. In this sense, it is still abstract and can also be regarded as an ‘artist impression’ of the designated solution. It does not provide details on specific servers, for example, or specific numbers, as will be specified in a technical design. But it does provide an interpretation of the technical components that will be applied for (shared) use. Note: (Part of) the relationships with adjacent functions/services have not been included in this simplified Design Outline. In an actual Design Outline, these would be highlighted, as they represent the added value provided by the practice of architecture, i.e. the coherence between the construction elements and the environment in which this construction functions.
The application of the workflow, as described above, is of increasing importance in a time in which infrastructure is becoming more versatile and powerful, but also increasingly complex. For the consumers of infrastructure services, it is important to define the required functionality and quality, and to determine the manner in which this functionality must be implemented, and – as a consequence - the guidelines that will apply.
In addition, more and more infrastructure is being purchased as a product (Cloud). This decreases the autonomy that many construction-oriented infrastructure architects and senior infrastructure designers have until now been enjoying in selecting the resources, generating the technical design and in realizing the final solution (building and constructing). Whereas the architecture of the solution was long regarded as an implicit fact (which was part of the expertise that experienced designers and constructors could apply easily and without much ado), it is now necessary that the party requiring the functionality and the party providing the functionality have a clear and unambiguous discussion on this matter, without a priorientering into a discussion on the technical aspects. What we do want, however, is an insight into the functionality and the quality of the products we select and purchase.
In other words: the architecture and the architectural choices must become clear. The DYA|Infrastructure Repository contains the basic models for this as a starting point for these discussions. It is the first step towards a vocabulary that lives up to the functionality and the quality that infrastructure landscapes provide. And this is a good basis for increasingly improving the spatial planning of our digital landscapes.
1 ArchiMate® [The Open Group, 2009] is used for the presentation of the models, whereby the (probable) availability of the Infrastructure functionas behavioural element in the forthcoming version 2 of the language has been anticipated. A modelling guide in which the rules for the combined use of ArchiMate® and DYA|Infrastructure have been defined and provided with good practices will soon be made available in the DYA|Infrastructure Repository.
3 For a complete development of the pattern variant shown, refer to the definition in the DYA|Infrastructure Repository