SOA Transformation through SOA Upgrade
Much has been said about Oracle SOA Suite 10g (or JCaps) upgrades to 11g and how features map between both versions. There is also plenty of information online about this topic both official and unofficial. It’s not news to many that for example SOA Suite 10g is currently in extended support and product will enter sustaining support by the end of 2014 (I will explain more about what extended and sustaining support means later in the blog). However one fact remains truth: There are still many companies out there running platforms that are (or soon will be) in sustaining support, and that don’t yet have an upgrade strategy. I say this based on my own experience as I am currently helping several customers do exactly this.
Having said that, I wrote this blog in an attempt to give SOA experts, Integration Leads and Architects key pointers that can serve as inspiration to come up with a transformational approach when defining an upgrade strategy. Note that I am using the word “transformation” deliberately and I will explain why shortly.
Note that although this article is mainly related to the Oracle SOA 10g to 11g technology stacks, the approaches, tips and information provided in this blog should also be applicable when defining any technology upgrade. In fact, once 12c is more mature I will probably refresh this blog to cover 11g to 12c upgrades.
Following my key pointers to help you define your upgrade as a SOA Transformation:
1) Understand the product roadmaps and planning to move in advance
2) Take a SOA Transformation approach and not just a technology upgrade
3) Elaborate a SOA Transformation Roadmap
4) Understand current and future technology stacks and identify potential risks and challenges in advance
5) Define a service transformation methodology
6) SOA transformation also requires organisational changes and maturity
1) Understand the product roadmaps and planning to move in advance
This is one of the most important points and one that many have either failed to understand or have just ignored (hence why many companies still stuck in 10g and have no plan to upgrade yet). This is important because by understanding the product releases and features, release dates, and support lifeline you can plan in advance an upgrade approach and avoid having to do something tactically, in a rush and with limited budget.
Before getting further into this topic, it is critical that you understand the basics of Oracle product releases:
Based on this diagram and the information described in earlier points I would strongly recommend that:
Let’s say for example that you are looking at upgrading your SOA infrastructure to a new major release, let’s say 10g to 11g or 11g to 12c. A typical upgrade project would focus mainly on technology and would almost certainly aim at changing as less as possible the existing code and services. By simply doing a technology upgrade, one would not only miss the opportunity to leverage the new product features, product enhancements and latest best practices available (because you would try migrate the code as-is whenever possible) but would also miss an opportunity to consolidate, rationalise and improve your existing services landscape therefore bringing benefits that would result in lower TCO.
For this reason, instead of focusing merely on the technology shift, take a step back and look at the bigger picture. Think about the potential benefits that you could bring to your organisation if you could also transform (not update or improve but transform) the processes and organisational/cultural aspect of the current solution that prevents realising such benefits.
How to know what benefits can be realised? There is always room for improvements. There will surely be areas that could be improved and haven’t for different reasons. For example there is no budget or there is no sponsorship from above to make the necessary changes or simply there is no enough skills in-house to execute such changes. To identify such areas that could be improve best thing to do is to conduct an assessment of your current integration landscape as suggested in point #3.
Also because a transformation approach requires fundamental changes to the core processes, technology and organisational/cultural aspects of your SOA solution, it is imperative of success that you engage with different parts of your organisation as early as possible to sell your approach and get buy-in. The toughest part of a transformation project is nothing to do with technology, it’s in fact dealing the the organisation and cultural changes it brings. I elaborate further on this topic in point #6.
Last but not least, there is no project is there is no funding. So to secure funding you must get approval from CFO or the like (unless there is spare IT budget to spend on this, in which case you would still have to justify the budget needed to CIO or CTO). I always compare this to going to a bank and asking for a loan. The bank wont lend you any money unless they are sure that they will get the money back and with interests (benefits). The bank business is to lend you money, so they want to do it, but they qualify the risk and if it makes sense then they will do it. Is the same within an organisation. C-level and D-level executives want to fund projects that deliver tangible benefits. But they wont just give funding away without proper analysis that there is a return on any investment they make. So to prepare for this you usually need to:
This may sound a lot more simple than it actually is in practice. In order to succeed in elaborating a roadmap you must understand:
This is a task that you will have to elaborate into detailed during your foundation phase. However even before you get to this phase, you should already have a good understanding of your as-is technology stack and how it maps to your to-be technology stack. Following an example of what I mean. The diagram shows how Oracle SOA Suite 10g and related technologies maps to the 11g suite and related technologies.
There are several factors that you should understand before you even start putting together a plan. What you are looking for is factors that will drive higher risks and effort. For example you should be looking for understanding:
It doesn’t have to be an overkill methodology. What you need is some concise steps which sets in stone the approach to follow when transforming a service from your existing platforms (i.e. 10g or other legacy) into your target solution. Purpose of the method is to ensure that all transformation activities are occurring following a consistent method and using the same set of guidelines and standards.
6) SOA transformation also requires organisational changes and maturity This is for me without a doubt the most complex aspect of a transformation initiative. This because you can change all tools and even upgrade most of your processes , however without having a behavioural and cultural change in your organisation chances are that the same issues you had in the past will repeat all over again. This is explained in detailed in my SOA Magazine article 9 Tips for Organizational Maturity in SOA also available in my blog www.soa4u.co.uk
Having said that, I wrote this blog in an attempt to give SOA experts, Integration Leads and Architects key pointers that can serve as inspiration to come up with a transformational approach when defining an upgrade strategy. Note that I am using the word “transformation” deliberately and I will explain why shortly.
Note that although this article is mainly related to the Oracle SOA 10g to 11g technology stacks, the approaches, tips and information provided in this blog should also be applicable when defining any technology upgrade. In fact, once 12c is more mature I will probably refresh this blog to cover 11g to 12c upgrades.
Following my key pointers to help you define your upgrade as a SOA Transformation:
1) Understand the product roadmaps and planning to move in advance
2) Take a SOA Transformation approach and not just a technology upgrade
3) Elaborate a SOA Transformation Roadmap
4) Understand current and future technology stacks and identify potential risks and challenges in advance
5) Define a service transformation methodology
6) SOA transformation also requires organisational changes and maturity
1) Understand the product roadmaps and planning to move in advance
This is one of the most important points and one that many have either failed to understand or have just ignored (hence why many companies still stuck in 10g and have no plan to upgrade yet). This is important because by understanding the product releases and features, release dates, and support lifeline you can plan in advance an upgrade approach and avoid having to do something tactically, in a rush and with limited budget.
Before getting further into this topic, it is critical that you understand the basics of Oracle product releases:
- Oracle product releases: There are two main points that you need to be aware of when talking about releases: Major release and patch set release (also referred to as component release). Let’s take for example Oracle SOA Suite 11gPS6 which is the same as saying 11.1.1.7
- Oracle talks about product versions in two different ways: 1) Product release name (i.e. 11gPS6) , 2) Product release number (11.1.1.7.x). Either can be used to refer to a specific product release, however I prefer the release number as it gives more detail.
- 11 is the major release. so for example 10g to 11g actually means major release 10 to major release 11. A major release means that there is a significant change in technology platform. It usually also means that you can’t simply apply a patch but rather requires more planning and preparation as it might require having to install and configure separate infrastructure and more elaborated service/code migration
- The “g” is for “grid”. This is more of a commercial/marketing thing. For example in version 12, it’s 12c where “c” is for cloud
- PS standards for patch set. A patch set is a single patch that contains a series of patches. In the example discussed PS6 standards for patch set six
- For details on how to read the product release number refer to: http://docs.oracle.com/cd/E28280_01/core.1111/e10105/release.htm
- Oracle patching and upgrade terminology: Oracle has specific definitions for what patching, migration and upgrade means. Patching is basically applying a patch to an existing installation. Migration is moving code from a third party product to an Oracle product. Upgrade means moving from one major version to another (i.e. 10g to 11g, or 11g to 12c). For more info on this refer to following link:
- Oracle Installation, Patching, and Upgrade Terminology: http://docs.oracle.com/cd/E28280_01/doc.1111/e16793/overview.htm
- Oracle support stages: There are 3 support stages:
1) Premier Support: This is where you want and should target to be as full support will be provided in terms of patching, security fixes, better product certification with other products and less obvious but important there will be more people skilled in support teams to deal with products in this stage
2) Extended Support: Pretty much the same as premier support apart from the fact that you will have to pay an extra fee (on top of standard fee). Also extended support may not include certification with some new third-party products/versions
3) Sustaining Support: You should always plan to avoid going into sustaining support. Reason is simple: not only an extra fee will be require on top of standard fees, but Oracle will not create new patches or fixes for products in sustaining support. Moreover there will be less available Oracle support resources to help you with you issues as most of the engineers are already supporting latest version of the product (versions in premier and extended support)
Refer to the following documentation for more details on Oracle support stages and support policies:
- Oracle Lifetime Support Policies main page: http://www.oracle.com/us/support/lifetime-support-068561.html
- Fusion Middleware Support Policy: http://www.oracle.com/us/support/library/lifetime-support-middleware-069163.pdf
Based on this diagram and the information described in earlier points I would strongly recommend that:- Plan your technology upgrade to latest major release at least one year before extended support starts. This will ensure that you will always be in premier support
- Try to keep up with latest patchsets. Note that there are constraints in extended and sustaining support around which patch set you are running (refer to previous links). Note that normally patchsets do not require re-coding and all migrations are usually assisted and can even be scripted. If you are worry about regression testing your code, look into implementing continuous integration or similar disciplines to automate execution of core test scripts (this is a best practice and you should try to do it anyway)
- If you already too late for you meaning you are in extended support (i.e. running 10g) and have no upgrade planned or you are only planning upgrade at this point, then by any means try to do it properly just don’t go and rush it. Chances are that you may anyways end up having to run the platform in sustaining support, not ideal by any means, but also not the end of the world. So my suggestion would be don’t waste any more time and plan a transformation strategy (see my subsequent points)
But what does it mean to take to take a SOA transformation approach?
Let’s say for example that you are looking at upgrading your SOA infrastructure to a new major release, let’s say 10g to 11g or 11g to 12c. A typical upgrade project would focus mainly on technology and would almost certainly aim at changing as less as possible the existing code and services. By simply doing a technology upgrade, one would not only miss the opportunity to leverage the new product features, product enhancements and latest best practices available (because you would try migrate the code as-is whenever possible) but would also miss an opportunity to consolidate, rationalise and improve your existing services landscape therefore bringing benefits that would result in lower TCO.
For this reason, instead of focusing merely on the technology shift, take a step back and look at the bigger picture. Think about the potential benefits that you could bring to your organisation if you could also transform (not update or improve but transform) the processes and organisational/cultural aspect of the current solution that prevents realising such benefits.
How to know what benefits can be realised? There is always room for improvements. There will surely be areas that could be improved and haven’t for different reasons. For example there is no budget or there is no sponsorship from above to make the necessary changes or simply there is no enough skills in-house to execute such changes. To identify such areas that could be improve best thing to do is to conduct an assessment of your current integration landscape as suggested in point #3.
Also because a transformation approach requires fundamental changes to the core processes, technology and organisational/cultural aspects of your SOA solution, it is imperative of success that you engage with different parts of your organisation as early as possible to sell your approach and get buy-in. The toughest part of a transformation project is nothing to do with technology, it’s in fact dealing the the organisation and cultural changes it brings. I elaborate further on this topic in point #6.
Last but not least, there is no project is there is no funding. So to secure funding you must get approval from CFO or the like (unless there is spare IT budget to spend on this, in which case you would still have to justify the budget needed to CIO or CTO). I always compare this to going to a bank and asking for a loan. The bank wont lend you any money unless they are sure that they will get the money back and with interests (benefits). The bank business is to lend you money, so they want to do it, but they qualify the risk and if it makes sense then they will do it. Is the same within an organisation. C-level and D-level executives want to fund projects that deliver tangible benefits. But they wont just give funding away without proper analysis that there is a return on any investment they make. So to prepare for this you usually need to:
- Define a strong business case: The business case should express qualitatively and quantitatively why funding this initiative is the right thing to do. It should emphasize on the cost of doing nothing or doing it wrongly. Furthermore the business case should not be based solely on the fact that your platform will run out of premier or sustaining support at some point. Yes this will certainly help, however to justify a transformation project funding you need a lot more substance. If you have some budget available and you don’t have the time to do this, you could get an expert to help you define the business case. Below some references that can serve as inspiration for this task:
- http://www.wikihow.com/Write-a-Business-Case
- SOA Governance for the business: http://www.soa4u.co.uk/2013/11/oracle-soa-governance-for-business.html
- To support your business case, define a SOA transformation roadmap which lists at a high level (but with enough substance so it looks thought through) the set of activities needed to get to where you want to be. Following point elaborates more on this topic.
This may sound a lot more simple than it actually is in practice. In order to succeed in elaborating a roadmap you must understand:
- Current integration landscape including architecture, maturity and service catalogues (see previous points)
- Ensure that you clearly understand your current integration landscape by creating a service catalogue. I am not only talking about your 10g services, but rather all your integration services and interfaces. It is very likely that you still have point to point interfaces written in PLSQL or using some old platform which has now become legacy. Some resources that can help you with this task:
- Excellent article providing 7 tips to elaborate a service catalogue: http://searchsoa.techtarget.com/photostory/2240212819/Seven-steps-to-develop-an-IT-service-catalog-Template-for-success/1/Using-a-service-catalog-to-aid-in-IT-documentation
- Excellent article by Manuel Rose and André de Oliveira Sampaio that talks about other things about how to create service catalogues blueprints with Oracle Enterprise Repository (OER): http://www.oracle.com/technetwork/articles/soa/rosa-sampaio-soa-gov-2080776.html
- Conduct an assessment of your current maturity state and then define a target maturity level. The target maturity state should be based on archiving tangible IT and business benefits. This is very important so you understand clearly where you stand and where you want to be. If you don’t understand where you stand and decide to upgrade your platform to 11g, even though you will be in a supported platform, the issues you have in your 10g implementation will remain there. Some resources to help in this task:
- Chapter 2 of my book Oracle SOA Governance 11g implementation talks about this : http://packtlib.packtpub.com/library/oracle-soa-governance-11g-implementation/ch01lvl1sec10#
- Oracle SOA Maturity Model: http://www.oracle.com/technetwork/topics/entarch/oracle-wp-soa-maturity-model-176717.pdf
- SOA Readiness Assessments: https://blogs.oracle.com/reynolds/entry/soa_readiness_assessment
- Based on the outcome of the assessment define some requirements that would have to be addressed in order for your implementation to get to the defined maturity state.
- Then plan to deliver your requirements in phases. The initial phases should focus on defining where you want to be based on where you are and setting up a strong foundation and governance framework that will ensure success once you start implementing the services. A high level view of this plan is basically your Roadmap. Some resources to help with this task:
- http://www.soa4u.co.uk/2014/01/soa-governance-faq.html
- SOA Governance for the business: http://www.soa4u.co.uk/2013/11/oracle-soa-governance-for-business.html
Below a sample high level view of a roadmap.4) Understand current and future technology stacks and identify potential risks and challenges in advance
From this sample high level roadmap I would like to highlight the following:
- There are to major phases: Foundation Phased and Implementation Phase. Foundation Phase –as the name suggests, aims to deliver a strong foundation. Key design decisions, standards, reference architectures, enhanced software development lifecycles, methodologies and governance frameworks should be defined delivered in this stage. Implementation Phase focuses in the rollout of the new platform not only to existing projects that will have to be upgraded or migrated from other platforms but also by making the new platform available to incoming projects
- Implementation phase starts with a pilot project. This is not a PoC but rather a real project with very limited scope but that use cases covers enough scenarios and test cases to ensure that the future architecture is robust enough to support known requirements
- Once the pilot has been delivered a “lock down” should be defined to the as-is platform. For example if you are running 10g, you should not allow any more developments in the 10g platform once lock down has been put in place. Again this is easier said than done, and politics will kick in at this point specially as some influential and important projects will demand a solution
- Transformation of existing projects to the new 11g platform doesn’t have to be and should not be in a big bang. You should define a plan that allows you to run your 10g or other platforms in parallel to the new 11g platform so you can incrementally align projects which will –in a phased manner, implement the same services in your new 11g platform
- New incoming projects will require support. Lots of support. Don’t just assume that providing a link to some documentation will be enough for them to understand how to deliver according to the to-be reference architecture, guidelines and frameworks. One way of dealing with this is to assign some resources (ideally architects or designers) from your team into each of these incoming projects. These resources should act as mentors of the new solution and should coach on how to do things the new way
- This won’t take a day and it will require planning and it will require proper sponsorship from the right level (I keep saying this and sometimes I feel that I sound like a broken record, however it’s so important that I don’t mind repeating it)
- The following link provides some good tips when executing transformation programmes: http://www.itbusinessedge.com/slideshows/show.aspx?c=85374
This is a task that you will have to elaborate into detailed during your foundation phase. However even before you get to this phase, you should already have a good understanding of your as-is technology stack and how it maps to your to-be technology stack. Following an example of what I mean. The diagram shows how Oracle SOA Suite 10g and related technologies maps to the 11g suite and related technologies.
There are several factors that you should understand before you even start putting together a plan. What you are looking for is factors that will drive higher risks and effort. For example you should be looking for understanding:
- Upgrade paths: This no only applies to code but also to the code. For example, you should be aware in advance if there are specific restrictions in terms of minimum versions and patch sets that your platform and code should be on before doing an upgrade. You should also be aware of there are assisted ways of upgrading your core infrastructure as well as your code. In the previous diagram I depicted how some technology components of 10g maps nicely to 11g and how others don’t. For example, BPEL 10g and ESB 10g maps directly to SOA Suite 11g BPEL and Mediator engines. It is also visible that there is an assisted / automated way of upgrading the technology and code. However it is not the same story for other components such as OBPM 10g, BPA 10g processes and WSM 10g. These will require manual re-coding (yes re-coding as there is no official wizard or tools am aware of to automate this). Some good information on this (specially on the topic of 10g to 11g):
- A good 10g to 11g upgrade presentation. http://www.slideshare.net/oracle_imc_team/oracle-soa-suite-upgrade-10g-to-11g
- Oracle Upgrade Guides: http://docs.oracle.com/cd/E28280_01/upgrade.1111/e10127/toc.htm
- Upgrading business rules: http://docs.oracle.com/cd/E21764_01/upgrade.1111/e10127/migrate_rules_app.htm#FUPSS915
- AIA Migration: http://docs.oracle.com/cd/E29542_01/doc.1111/e17361/ch7.htm
- Architectural and best practices : Even if there are guided upgraded paths for some technology components there are several other factors that should be consider if you truly want to embark on a SOA transformation and not just a technology upgrade. Don’t just use a wizard because they are available (also be warned about Wizards –see the following point). To truly transform, you should evaluate architectural factors as well as new best practices available. Ideally a foundation phase as previously described would target to set in stone your future reference architectures and best practices so this task become a lot easier. Based on the previous diagram I can provide several examples of what I mean (however you should come up with your own depending on your target architecture and best practices):
- According to Oracle best practices, the strategic service bus in 11g is OSB and not Mediator. However if you have built services using ESB 10g the easiest way to migrate the code to 11g is by using the assisted wizards or tooling meaning that your ESB code will end up in SOA composites which will run in Mediator engine. This however doesn’t mean that you are making the best out of your to be architecture because in 11g for example, OSB has better routing capabilities. However in some scenarios migrating directly to Mediator might make sense. For example, if your service implements WS-Addressing (asynchronous services) as OSB does not supports this natively (there are work around thought but it’s not native).
- This is a very good article by Guido Schmutz describing best practices on when to use OSB: http://www.oracle.com/technetwork/middleware/soasuite/learnmore/where-and-when-to-use-osb-1885528.pdf
- Technology constraints: There could several changes in the technology stack that are less obvious and you will only find out after you start testing your code. You should try and identify this either by learning from other’s lessons learnt or engaging your vendor for detail feedback on this. Again using the previous diagram, I can list the following examples of what I mean:
- Wizards don’t to it all: Even for the technology components where an upgrade wizard is available don’t just assume that everything will work once code has been upgraded. There are several things these wizards don’t take care off. I would recommend the following link for more details on this: http://www.slideshare.net/oracle_imc_team/oracle-soa-suite-upgrade-10g-to-11g
- There is no MDS in 10g. Therefore if in 10g you had all of your schemas deployed as web application and accessible via HTTP ideally you would like to move these schemas into MDS in 11g (you don’t have to, but this goes back to my point about best practices). This however would require some changes in the code (so mds references are used instead of HTTP references) and also the schema project itself (so it can be deployed to MDS via JDeveloper or ANT). Refer to following link for more info: http://redstack.wordpress.com/2011/02/28/deploying-your-canonical-data-model-to-mds/
- If your 10g services consume other 10g services is by using concrete WSDL rather than abstract WSDL’s then you will run into issues. In case you don’t know the difference between the two, abstract WSDLs are usually generated by JDeveloper and do not contain any binding and/or port type information. Once a JDeveloper is deployed, Oracle SOA Suite automatically generates a concrete WSDL that actually refers to abstract WSDL but that defines the binding protocols and port types. In 10g during start up Oracle Application Server started each service in the same order that they were deployed. This ensured that if you had deployed your services in the right order, then they should all start correctly. However in 11g, Weblogic Server starts the services randomly so if a particular composite depends in a concrete WSDL for a service that hasn’t yet started, then the service will fail to start. To fix this issue you should change your code so it refers to abstract WSDLs whenever calling another internal service. More on this: https://blogs.oracle.com/aia/entry/aia_11g_best_practices_for_dec
- OBPM 10g (former ALBPM) doesn’t have assisted upgrade toolkit to 11g. So if you want to migrate OBPM 10g code to 11g you have to re-implement your processes. There is a chance that OBPM 10g will have assisted upgrade tooling directly into BPM Suite 12c
- WSM 10g also doesn’t have assisted upgrade toolkit to 11g. So if you want to upgrade WSM agents you should probably be looking at manually re-implementing them as 11g WSM policies and if you want to upgrade your WSM Gateways you should manually re-implement 10g policies into Oracle API Gateway (OAG)
- Business Process Architect (BPA) Suite also doesn’t have assisted upgrade toolkit to11g. Some aspects of what BPA suite can do can be migrated to BPM Suite 11g. However more richer enterprise modelling features Value Added Chains, organisational modelling, and others will only be supported in BPM Suite 12c
- Oracle Application Server 10g is no longer use in 11g and beyond. Weblogic Server becomes the core Oracle’s strategy JEE server
It doesn’t have to be an overkill methodology. What you need is some concise steps which sets in stone the approach to follow when transforming a service from your existing platforms (i.e. 10g or other legacy) into your target solution. Purpose of the method is to ensure that all transformation activities are occurring following a consistent method and using the same set of guidelines and standards.
6) SOA transformation also requires organisational changes and maturity This is for me without a doubt the most complex aspect of a transformation initiative. This because you can change all tools and even upgrade most of your processes , however without having a behavioural and cultural change in your organisation chances are that the same issues you had in the past will repeat all over again. This is explained in detailed in my SOA Magazine article 9 Tips for Organizational Maturity in SOA also available in my blog www.soa4u.co.uk
Nice article although one point to note is that the assisted upgrade wizard for 10g ESB/BPEL processes does not take account of everything, i.e. the rewrite of BAM with the introduction of 11g. Hence i would suggest caution with the wizard as this should not be seen as a quick win in order to get onto the 11g platform, more of a stepping stone to understand how 11g views the 10g projects.
ReplyDeleteThat is absolutely true hence my statement "Don’t just use a wizard because they are available. To truly transform, you should evaluate architectural factors as well as new best practices available. ". I will however update the blog to provide a "warning on the wizards" hope things are well at P4U :)
Delete