Application Management Services (AMS)

Avant Application Management Services (AMS) offers a suite of services that span the full spectrum of Application Lifecycle Management (ALM), including governance, development, and maintenance.

Through the use of modern development processes, tools, and methodologies, we help organizations free up valuable resources, while lowering total cost of ownership, improving service levels, and streamlining processes.

From corrective and routine maintenance, to legacy application modernization, our experience in managing mission-critical enterprise applications and bridging the gap between business management and software engineering makes Avant your perfect partner.

Contact us today to learn more about our Application Management Services

Systems Integration Services

Avant Systems Integration Services are focused on delivering integrated solutions built on best-of-breed technologies.

Using modern software and project management methodologies we provide requirements definition, solution architecture, implementation, customization, training, and support to deliver complete solutions to your business problems.

Out clients grow their businesses on our foundation of technology experience and alliance partners.

Contact us today or visit our Partners page to learn more about our Systems Integration Services

Application Development Services (ADS)

Avant Application Development Services (ADS) are build on decades of experience in delivering robust enterprise grade software solutions. Our application development consultants are experts across a broad range of technologies, architectures, platforms and devices. By employing the principles of Agile and DevOps we leverage modern development processes, tools, and methodologies to deliver high quality custom solutions.

Through all stages of the Software Development Life Cycle (SDLC) from requirements, design, test, to implementation, our development team is here to support your organization.

Contact us today to learn more about our Application Development Services.

Simplified Issue Linking in JIRA

Issue Linking has proven to be a very powerful feature for teams using JIRA. However, as an Atlassian Expert Consultant who often works with newer users or teams that are just starting to use JIRA, I find that the Issue Linking feature is not as prominent or easily accessible within the UI as it should be. I’ve worked with JIRA users who have been using the software for years who still had not stumbled upon the Issue Linking features.

In a recent client engagement, the organization’s proposed JIRA structure (following Scaled Agile Framework methodology) necessitated heavy use of Issue Linking. The problem I was faced with was how to prevent users from creating incorrect Issue Links between certain Issue Types, all the while ensuring that the user experience remained as simple as possible. Having to always use the More drop-down to access the Link dialog was problem enough (well at least for the first link for an issue), but on top of that, the user was then confronted with a whole slew of different link types (inward and outward) and would need to be relied upon to know what selection needed to be made in order to convey the correct reason for the linkage.

What I really wanted was a custom field that I could label appropriately, enter multiple issues keys into and automatically create the appropriate issue link using the correct issue link type (whether it be inward or outward). After some extensive reading/research into marketplace add-ons that could help me accomplish this and some deep thought, I came up with what I think is a perfect solution!

Finding the right Issue Picker

If my idea was going to work, I needed to identify a way to allow a user to enter multiple JIRA issue keys into a single field. Restricting the entry to just one issue wasn’t the user experience I was hoping to accomplish. Unfortunately, JIRA does not come with a built-in Issue Picker custom field. While there are a number of Issue Picker fields available as add-ons within the Atlassian Marketplace, a free add-on Query Issues Custom Fields add-on for JIRA Server fit the bill perfectly. Not only does this add-on provide a custom field that allows for multiple JIRA keys to be entered at once, it also provides the ability to restrict which keys can actually be entered into the field via a JQL filter. On top of that, the custom field allows for auto-complete functionality which was icing on the cake.

Configuring the Issue Picker

Once, the Query Issues Custom Fields add-on was installed, creating the custom field was fairly straightforward. You must first add the custom field, which you will find in the Advancedsection. This add-on provides three custom fields, for this post we will be focusing on the Query Issue Multi Linker Custom Field type. Once selected, name the field appropriately. In this post I will demonstrate how to set up a field that automatically creates “is blocked by” issue links, so this field will be aptly named “Blockers”. When presented with which screens to add the new field to, select the appropriate Create screen(s) but no others for now.

Once the field has been created, we can now configure the field using the add-ons configuration screen. The location of the configuration screen is non-standard. You can find it by going to the System page, and then locating the Configure Query Issue Custom Fields link on the left hand navigation menu. You will then see your newly created custom field within the Multi Linking Fields table.

Clicking on the Configure action link, pops up a dialog where you can restrict the JIRA keys that can be entered into the field via JQL (here I’ve restricted the keys to a single project), add auto-complete functionality, as well as providing some JIRA issue field data to the user as they enter keys. For this example I have selected: Key, Status, Assignee, Due, Priority.

Once you click apply, you are brought back to the table that summarizes your fields configuration.

The field is now setup for use, but we now have to tie it into the workflows that we want it to operate with.

Automating the Linking Process

The first step in automating the issue link creation process is to identify which workflows will be using the newly created issue picker field. For this example I am using one of the built in JIRA workflows and have added two post functions to the Create transition which I’ve highlighted in the workflow diagram below.

The first post function to add is the Create Issue Link function that is bundled with the JIRA Workflow Toolbox add-on for JIRA Server. Once this add-on is installed in your JIRA instance, the post function will be available for you to select.

The available configuration options for the Create Issue Link post function are vast, but for this example the setup is extremely simple and straightforward. There are three sections that you need to be concerned about: Issues at source end, Issue Link Type, and Issues at destination end.

For the Issues at source end configuration, simply leave the default selection of “Current Issue”. This tells the post function that the issue being created is the Inward link, or in this case, the issue that is being blocked.

For the Issue link type configuration, select the type of link that you want to create. For this example, we want the “is blocked by” issue link type which happens to be the default selection.

For the Issues at destination end configuration, select the Issues in field option, and choose the custom field that you created earlier. For this example we will select the custom field we created earlier named Blockers.

Once you have completed the above configuration and have added the post function to the transition, you will see an entry like below.

Next, we need to add another post function, Clear Field Value which is available via the JIRA Suite Utilities add-on. If this step gets missed then there is potential for the custom field to appear on the issue view screen. We don’t want that though. We only want to use the custom field as a temporary holder of JIRA issue key data so that the workflow post function that creates the issue links can do its work. Once the links are created the Blocker custom field should be cleared.

Once you have added the two post functions, your list of post functions for the Create transition should look similar to the screenshot below. Notice that the “Creates the issue originally” built-in post function has been moved up to the top of the list. This needs to be in that order otherwise the issue link creation will not occur.

The configuration is done. Ensure that you have published your workflow so that your JIRA users can take advantage of the new functionality!

Issue Linking Simplified!

Now when users create issues, they can quickly create issue links using the custom field that we created and the workflow post functions take care of the heavy lifting. Users, no longer need to know which Issue Link Type they should select and only Issues that should be linked to are allowed to be linked to. I’ve provided screenshots to compare the built-in behavior versus the Simplified Issue Linking behavior that I have provided you with in today’s post.

Built-in Issue Linking	Simplified Issue Linking

Both methods of issue linking, whether you use the built-in behavior or the simplified behavior, result in issue links getting created and displayed in the expected Issue Links panel.

Note: In order to simplify the example, I’ve only discussed using this custom field within the Create issue dialog. The same configuration process can be used within any workflow transition, or within a circular transition (source and destination status are the same). A custom screen could be created to only display this particular field during the transition, or you could have a circular transition display a dialog with multiple versions of the custom field (e.g Blockers, Blocks, Relates To, or any custom issue link type).

Jamie Saliga is a Senior Atlassian Expert Consultant at Avant, he has been using JIRA for more than 6 years and has vast experience in JIRA administration and training. Stay tuned for more great JIRA stories.

References:

Query Issues Custom Fields add-on for JIRA Server

JIRA Workflow Toolbox add-on for JIRA Server

JIRA Suite Utilities

Continuous Improvement with the Root Cause Analysis add-on for JIRA

Every company or organization strives for continual improvement and while companies developing software are often on the leading edge of this ideology, there can be some gaps that limit the potential for improvement which are often easily addressed. The strive for higher quality is one of the most often heard continuous improvement goals within a software development company and with good reason! Customers have higher expectations of software then ever before and will often not wait around for problems to be eventually addressed before moving on to the next competitors offering. What this inevitably means is that the cost of every bug that finds its way into production versions of our software is steadily increasing as our customers expectations increase. While every bug that makes its way out of a developers hands will have a cost, the ultimate cost of a production bug is the irreparable damage it can have on your customer relationships.

Addressing a quality problem within a software development team can be a tricky task since it is often not obvious what factors are contributing to the bugs that your customers are finding in your software product. With every bug uncovered and fixed many organizations have unknowingly missed a critical continuous improvement opportunity.

When a developer investigates a reported bug and implements a code fix, some form of Root Cause Analysis is performed in order to determine what the problem was and the most appropriate way to fix the root cause. More often then not, the insight gained from such an investigation is only beneficial to the developer fixing the bug. The developer and tester involved with the change that caused the bug often never hear about the problem and may continue to make the same mistake again and again.

With the Root Cause Analysis plugin for Atlassian JIRA, the Root Cause Analysis process becomes a more explicit part of your software development workflow and ensures that team members are aware of downstream issues that are a result of code changes that they had either implemented or tested.

Collaborative Analysis

Bring the whole team together for the Root Cause Analysis so that the continuous improvement experience can benefit all affected individuals. Once the bug investigation yields the root cause (carefully avoiding pin-pointing a symptom) and the JIRA issue that the code change was made under is identified, the Root Cause Analysis plugin jumps into action. This identified JIRA issue becomes the “Root Cause Issue” and the Resolver and Closer information is automatically obtained.

Invaluable Feedback

Upon identifying the Root Cause Issue, the Resolver and Closer of the Root Cause Issue are notified via email about a bug being found with a code change that they had been involved with in the past. They are then prompted to review the bug and provide a summary of their review.  We’ve provided a JIRA dashboard gadget that will serve as a visual reminder that you need to take some form of action within a Root Cause Analysis session. We all know how easy it is to forget about an email notification!

This reflection on past work and being involved in the downstream affect of an error provides an extremely valuable continuous improvement opportunity that might otherwise have been missed!

Uncover Trending Behaviors

With the Root Cause Analysis plugin you will now be able to spot problem trends as they are happening within your team in order to understand what needs to be done to improve for the future. Using the Root Cause Analysis reporting features, you can get an organizational view of all root causes identified between a set of dates, use the filtering features to see only root causes identified within a particular project, or drill right down to a particular team member to understand why the tasks they work on often result in edge case bugs. Perhaps you’ll discover that many of the bugs that reach your customers could have been avoided by providing some additional training to your developers and testers in your software’s problem domain.

The Root Cause Analysis plugin for JIRA can be purchased for JIRA Server from the Atlassian Marketplace. Try it today to stop fixing symptoms and start drilling down to the Root Cause!

Logs and Ops - Streamlining Troubleshooting and Management of a Hosted Atlassian Environment

Simplifying and streamlining troubleshooting and management is critical for getting the most out of your Operations team. Today I’d like to talk about some of the tools we use at Avant to help us do just that.

So Many Logs, So Little Time

A huge challenge with any online hosted service is working with logs, and extracting meaningful data from them.

Imagine an application with the following components running on a number of Linux VMs:

• nginx reverse proxy / cache
• Tomcat container running JIRA
• PostgreSQL database

This will generate a lot of log data to sift through once it hits production.

What is the alternative to logging on to N number of VMs and grepping through logs? The ElasticSearch ELK Stack (AKA ElasticSearch + Logstash + Kibana). This tool set has been a huge boon for troubleshooting efficiency. Searching and reporting based on logs has never been easier.

In our configuration each node forwards the logs for its role, plus intrusion detection logs to a centralized Logstash instance, which filters and outputs the results to the ElasticSearch backend. Another monitoring tool such as Nagios or Zenoss will also forward check results which will in turn be aggregated in ElasticSearch.

Now we have centralized logs that are quick and easy to search and correlate events. You can easily pre-configure dashboard that gather all the usual suspects in one place for troubleshooting (nginx error logs, Tomcat errors, slow database queries, etc).

Why Am I Still Doing This Manually?

Doing things manually has some major drawbacks: it is labor intensive, error prone, and you can only trust the people with the right skills to do it. When your Operations team has limited capacity, it is time to automate the routine, time consuming tasks.

Enter Rundeck.

This excellent tool does exactly what it advertises on the tin, it ‘enables self-service operations’. Combined with the ELK stack, an operations engineer can look at the logs, troubleshoot the problem, and follow up by restarting a failed service, without having to log on to a box or tail logs. Here are some of the things we use Rundeck for:

• Managing services (start/stop/refresh)
• Re-configuring JVMs (add debugging, change memory parameters)
• Initiating and scheduling backups and maintenance windows

Rundeck also serves as a centralized cron server – with the added benefits of a slick web interface for activity reports, as well as notifications. This gives visibility to the operations team on everything that is supposed to happen as a scheduled task.

So What Does This Mean To Our Customers?

In the end, we provide a very high quality hosting service. Our operations team has the tool set at it’s fingertips to proactively address issues before they become a problem. If a problem does occur, we can rapidly address it with our streamlined troubleshooting process.

Mistakes arising from human error are minimized with a library of tested, repeatable operations tasks. This means less risk of downtime and faster maintenance windows.

Eliminate Merge Nightmares with Git Essentials: Part Two

In part one of the Eliminate Merge Nightmares with Git Essentials post, we presented a scenario in which I just realized that because of my teams branching and merging strategy (or lack thereof) the timeline for my project deliverable was in jeopardy. In part two of the series, we will go more in depth about the automated branch merging schemes available with Git Essentials and how this functionality can help reduce the chances of you and your team having to confront such a scenario ever again!

Merge Hell? Never Again!

One of the primary reasons for working in an isolated branch is to ensure that none of the code for the feature(s) we are currently working on get into the mainline branch until the new feature(s) are in a deliverable state. As we witnessed in the problematic merge scenario discussed in part one of the post, this puts us at risk of potential headaches when we eventually merge our completed changes back into the mainline branch. With Git Essentials’ continuous integration and delivery tool, the potential for these merge issues are drastically reduced by using Bamboo’s built in automated merge features.

Branch Updater

When working in an isolated branch, development work does not stop on the mainline (master) branch. Every code change made by other project teams into the mainline branch or code changes committed to the isolated branch contribute to divergence between the two branches. Bamboo has a built in automated branch merging scheme called Branch Updater that seeks to eliminate the risk of such code divergence for the teams working within the isolated branchs.

As you can see from the Branch Updater diagram (Fig. 1), any changes made from within the mainline (master) branch will get picked up by Bamboo and automatically merged into the isolated branch in order to keep the isolated branch from diverging from the mainline, yet keeping the changes from within the isolated branch from making it back into the mainline branch until the feature set is completed. Optionally, the changes just get built together without any actual push occurring on success.

With the Branch Updater automated merge scheme activated for your branches, have confidence in knowing that you will get immediate feedback when a new change in the mainline doesn’t work so well in your isolated branch so you can begin to do something about it now instead of later. You’ll never again get that sinking feeling when pushing your isolated branch back to the mainline at the end of a project when a whole slew of merge issues present themselves.

You have better things to be doing with your time then figuring out how to fit your teams new changes back into the mainline at that stage of the project anyways don’t you?

Fig. 1

Gatekeeper

While the Branch Updater automated merge scheme is great for keeping your isolated branch up to date and effectively reducing the potential for downstream merge conflicts, the Gatekeeper automated merge scheme (Fig. 2) is a great way to keep the mainline branch updated with successfully built code changes made from your isolated development branch.

Whenever changes are committed to the isolated branch, Bamboo will combine these changes with the current mainline code base and attempt an automated build and test sequence. Optionally, if the sequence passes without any failures, the new changes committed to the isolated branch will be officially pushed into the mainline branch.

This automated branching scheme is very valuable when there are numerous development teams working in isolation off of the mainline (or development) branch. All successful code commits will immediately become available for any project teams who are using the Branch Updater scheme. This allows all teams to get immediate notification when there are any code compatibility issues.

The best way to use this automated merge scheme is when your team is working under a Test Driven Development methodology. Having the tests defined up front will ensure that the isolated changes do not get pushed back to the parent branch until a set number of tests are passing. After all, there was a reason for us to be working in isolation to begin with right?

Fig. 2

How Do I Get Started With Git Essentials?

With all of these key features built into the Atlassian Git Essentials product it’s a no brainer to get started now to immediately start improving your development team’s daily operational efficiency. Contact the Atlassian Experts at Avant today to get set up with these industry leading tools or for a more in depth demonstration on how these tools can be leveraged within your company to increase confidence in timelines and overall throughput for all of your software projects.

Eliminate Merge Nightmares with Git Essentials: Part One

It’s late in the workday and I’ve just checked in the last chunk of code that brings together all of the individual pieces of functionality for a new feature that my team and I have been working on for the last couple of weeks. As the team lead, this is the point in the project where I normally would have to prepare myself for potential merge hell. When starting a new piece of functionality, I create a new branch off of the mainline (master) branch to isolate our team’s changes from other unrelated development work, with the intention of keeping our isolated branch up to date with the changes going on in the mainline but because this is a manual process it often gets forgotten. Now I am back at the helm of a branch that has diverged from the mainline for a number of weeks, crossing my fingers that the merge goes through without a hitch. As anyone who has worked on a code base that multiple teams are actively developing against at the same time knows, rarely will this merge succeed without numerous conflicts. Unless you are exceptionally lucky! (in which case you should also buy a lottery ticket).

So… with my fingers crossed I issue the pull request from our isolated feature branch into the mainline branch where we will be releasing the product increment from in a few days time. Now, imagine the scenario where your team has been heavily refactoring an area of complex code for the last couple of weeks, in order to fit in a new piece of functionality and during the merge process you discover that another team has been doing similar work in the exact same area of code. Slowly, I uncross my fingers as the realization sinks in that I have encountered just that scenario and I’ll be unable to merge my team’s changes into the mainline branch without significant code changes and/or breakage analysis. In my mind, the planned delivery date is now in jeopardy…

With Atlassian Git Essentials, this scenario is a thing of the past…

I Need a New Branch Quick!

The concept of a daily code check-in has been around for a while and is a widely used development technique to keep all team members up to date with each other’s code changes to allow for better collaboration in highly dependent areas of functionality. The goal is to avoid having working code isolated locally on a developer’s workstation for any longer than a work day. Now, think about isolated branches and how they continually diverge from their mainline branches right from the point of creation.  Why do we not employ this same concept for our branches? The most often heard criticism of such a strategy is that creating branches is time consuming and there is too much overhead associated with them. Here is where the integration within the Git Essentials product line really shines!

With JIRA Agile, new Stash branches can be created easily and on the fly with a single click of the “Create Branch” link from within every single JIRA issue. This provides the development team with the flexibility to create branches whenever necessary without ever having to leave JIRA. Creating these lightweight branches in this manner automatically links the new branch to the JIRA issue. The JIRA Development Panel (See Fig. 1) within the issue allows for exceptional transparency of all activity within the new branch from a code perspective as well as any build/deployment activity occurring from within Bamboo.

Branch creation doesn’t get any better than that!

Fig. 1

Do I Have to Setup the CI Server Now?

In the past creating and maintaining new branches was often thought of as a time consuming process with a lot of associated overhead. In order to reap all of the benefits of long lived branches you would often need to get in touch with an administrator user who had the ability to add new branches to the automated build process. Even the act of creating and destroying a branch often involved getting users with a certain level of permissions involved every time.

With Git Essentials, you no longer have to bother your administrator colleagues to get going with lightweight branching strategies. As soon as a new branch is created in Stash or directly from within JIRA agile, Bamboo has the ability to automatically recognize the new branch and start the continuous integration process when any new code changes are added to the branch using the parent branches settings by default. Your new branches will be running all of the build and test scenarios that you would expect effortlessly!

While Stash offers the ability to automatically clean up and remove branches after successful merges, Bamboo has the ability to ensure that these lightweight branches never start to pile up and clutter up the code repository.

Developers are not usually very good at cleaning up after themselves after all!

Fig. 2

How Do I Get Started With Git Essentials?

With all of these key features built into the Atlassian Git Essentials product it’s a no brainer to get started now to immediately start improving your development team’s daily operational efficiency. Contact the Atlassian Experts at Avant today to get set up with these industry leading tools or for a more in depth demonstration on how these tools can be leveraged within your company to increase confidence in timelines and overall throughput for all of your software projects.

Keep an eye out for our second installment in the series, Eliminate Merge Nightmares with Git Essentials: Part Two which will provide some insight into how automated merge schemes built-in to Git Essentials will further reduce the chances of you and your team encountering the merge hell scenario!

How Avant Uses JIRA for Inventory Tracking

Great Minds Think Alike?

The recent blog post over at Atlassian about how they use JIRA for asset tracking has prompted us to talk about how Avant does the very same thing. The asset tracking use case does a great job showcasing how effectively and easily mapping real-world processes can be done in JIRA.

When building out the asset tracking project we wanted to address the following core requirements:

• Track all roles for both physical and virtual systems
• Current status and user assignments
• Integration with existing support processes
• High level dashboard view

Here at Avant we have a number of different sources for information on assets, but no solid way of tying it all together. This is where JIRA comes in. With the flexible dashboards, and easy workflow creation we now have a rock solid process for keeping track of all our assets with the critical metadata missing from all the other sources. Having an all-in-one dashboard that helps administrators know and understand the roles, critical URLs and production status is absolutely awesome.

The Nuts and Bolts Of Our JIRA Inventory

Agile Boards For Inventory

Our homegrown inventory tracking makes use of custom issue types for the type of asset (Virtual Machine, Desktop, etc). What we wanted was a simple view that captures the current state of all items currently being tracked. This is where the Agile board comes in to play.

We use a Kanban board and a simplified workflow to keep track of the current status of assets.

JQL queries are used to neatly categorize similar assets into swimlanes. Once items have been decommissioned, they will drop off the board after an arbitrary amount of time (we like two weeks).

The Inventory agile board provides a one-stop-shop to view the current status of all assets being tracked.

The Unified Inventory Dashboard

Knowing the status of particular assets is great, but what about all the other important data admins and users might need to know? That is why we built a dashboard that captures the following important information:

• Your own hardware
• Pie charts showing overall breakdowns by platform, type and status
• Critical remote management URIs
• Who is assigned what desktop/mobile devices
• What role does the device fulfill

For example, you can quickly see what roles the various bits of virtual infrastructure play with this handy set of filter results:

Future Horizons

One area we will be looking to improve upon in the future is automation. Being able to have information collected from various systems that capture configuration and host info will be critical for streamlining the process, as it is currently 100% manual. Having a shop that has mixed Solaris, Linux and Windows means pulling from lots of different datasets, such as Active Directory, Puppet and LDAP. The great thing about JIRA is how easy it is to build the ‘glue’ to make all this happen with the REST API.

Keep an eye on this page for future updates on our “JIRA Inventory Tracking” adventure!

MadCap Flare Documentation Migration into Atlassian Confluence

Cool! We Have a New Collaboration Tool… Now What About Data Migration?

When a new tool is introduced into an organization with the intent of replacing an existing tool, there are many things to consider. Consequently, the process of moving the existing data from within the old tool into the new tool is not always at the forefront of such considerations and is most often written off as a trivial procedure that does not require much effort or thought. Existing tools are always well entrenched into an organizations current workflow and the result of such an entrenchment means that there is usually a massive amount of data that must be migrated into the new tool. This data currently residing in the existing tool is usually of great importance and an organization would want to be ensured that all of the historical effort expended to build up such data would not have been wasted. Requiring an organization to start with an empty data set after the switch to a new tool is usually not an option. Recently, we at Avant were presented with a client who intended to retool their software documentation knowledge base in order to allow for better collaboration on their existing documentation and to move away from a static/unchanging documentation model. This client decided that Atlassian Confluence was their new tool of choice. The idea was that once the existing documentation was migrated into the client’s Confluence instance, all employees would be able to collaborate instantaneously on the documentation in order to make it an ever-evolving knowledge base that better reflected the current state of the software that it documented at any point of time. Any content that was obscure or incorrect could easily be modified as encountered instead of the historical process of having to log an issue and wait for a colleague with the ability/know-how to modify the documentation within the Mad Cap Flare editor and subsequently publish the updated version for the rest of the organization to view. Moving to Atlassian’s Confluence offering was an easy choice for our client but migrating the existing documentation into Confluence from within Mad Cap Flare was proving to be extremely problematic. After the client performed some initial research on import options available within Confluence they had initially took it upon themselves to attempt the migration of approximately 350 tightly coupled pages via Mad Cap’s Microsoft Word export process. Confluence provides a simple Word document import functionality on a page per page basis but this “connector” was intended for text only which meant that all screenshots and icons within the original documentation would be lost during the import. It was quickly obvious to the client that this process was going to be very time consuming and would involve a large amount of manual work since none of the images shown within the exported Word document were visible within the resulting Confluence page, all links were no longer valid and things just generally didn’t look great. The whole point of moving to Confluence from Mad Cap Flare was to improve the documentation and to utilize the collaboration tools and functionality within Confluence to the maximum extent possible without having to rewrite the existing documentation. After failing to come up with a reasonable strategy to migrate their existing documentation from Mad Cap Flare into Atlassian Confluence the client came to Avant for our expert help and advice.

We Definitely Need an Automated Solution… But Where Do We Start?

When we at Avant were presented with our client’s migration situation we initially thought that surely somebody somewhere out there had come up against this problem in the past. Our initial analysis revealed an existing wiki conversion tool called the Universal Wiki Converter (UWC) which is developed and maintained by AppFusions to convert a variety of different wiki formats from original source markup into the expected Confluence markup. However, after thorough analysis of the UWC tool’s existing functionality/features and an exhaustive search of the discussions within the Atlassian Developer Portal along with other various message boards/blogs we came to the conclusion that nobody anywhere has efficiently migrated a complete documentation set from Mad Cap Flare into Atlassian Confluence without an immense amount of manual intervention, quite possibly performing every step of the process of such a conversion manually. Our client was not the first to come up against this challenge as there were numerous questions about how one would perform such a migration on many of the popular message boards and blogs relating to wiki migration across the internet. None of these conversations yielded anything of substantial use. There appeared to be some Perl API’s available to do simple HTML parsing but nothing that would provide for a migration that would result in a feature rich Confluence knowledge base that we at Avant wanted to deliver to our client.  High quality is what our clients expect when working with Avant. Turning back to the UWC tool, further analysis revealed that this tool was in fact open source which gave us the ability to acquire the source code and extend the tool to support the exact conversion that we required. By no means was this a straight-forward task but after gaining a thorough understanding of the tool’s source code (written in Java) along with the developer documentation available on Atlassian’s developer portal, we were able to add Mad Cap Flare to the list of supported wiki conversions by developing a multitude of Java based custom converter and builder extension classes. This effectively allowed us to focus on the conversion itself instead of having to put any unnecessary focus on interacting with the XML-RPC API exposed by Confluence which was the most enticing reason on our decision to settle on extending the UWC versus writing a new tool from scratch. The exported documentation set provided to us by our client was in a proprietary XHTML format and came along with all of the page referenced resources like images, pdf files, and video files (.swf, .wmv, .mov) necessary to rebuild the documentation in Confluence. A few critical XML files were located within the data which we required to fully understand the layout of the table of contents, the relationship between pages and the glossary items that rounded out the Mad Cap Flare output. We now had a base tool to work with and expand, as well as detailed knowledge of how the source documentation data was output and displayed by Mad Cap Flare. We were ready to start understanding what changes were necessary to be made to the UWC tool in order to reduce the manual effort down to as little as possible within the time we made available. We identified that the following tasks would need to be automated via the UWC tool in order for us to efficiently and effectively migrate the documentation into Confluence:

Page Referenced Resources & Content

The Mad Cap Flare documentation output was highly screenshot based and heavily referenced both internal and external resources. One of the main shortcomings we identified with the existing Word import process was that it offered no way to automate migration of any image, pdf or video resources included or referenced within a page. We definitely did not want to manually identify, upload and embed each individual resource on every page within the documentation set, this would have been an extremely time consuming process. Our automated solution needed to identify each and every resource that was referenced within a page and ensure that they got uploaded as attachments into Confluence when the UWC uploaded the resulting migrated page content. For image and video content, we needed to ensure that they would correctly appear inline within the page content. For PDF documents, we needed to ensure that the links would allow the PDF document to reference the correct attachment uploaded with the page. The UWC tool had existing converters for other wiki types that already automated this process but didn’t quite work for the XHTML used within the Mad Cap documentation export so a new converter class was created to account for these differences. The main difference we observed with Mad Cap images was that they used scripting to control the display of images which allowed for the higher resolution image to appear in a popup as the user hovered over the thumbnail with their mouse, while in Confluence, the embedded image can have a defined size while allowing the full sized image to be shown when the image is clicked on. Since we would only require one image, we would need to ensure that the high resolution image was uploaded to Confluence and embedded within the page as opposed to the thumbnail.

Table of Contents & Page Hierarchy

We knew that we needed to maintain the exact layout of the Table of Contents during the migration process but we didn’t anticipate the problem we would come up against in doing so. The UWC tool already had varied support for page hierarchy schemas that would work well for a variety of supported wiki conversions and at first glance we thought we would be able to leverage the existing FilePathHierarchy builder class. The documentation states that when utilized, this builder class will automatically create parent-child page relationships using the directory tree structure on your file system. Upon review of the file system layout of the html files within the Mad Cap export provided to us, this builder class looked like it would do the trick for our purpose. However, once we dug deeper into the file structure and compared it against the layout of the Mad Cap Table of Contents output it became obvious that there really was no parent-child page relationship based on the existing file system structure. Manually modifying the file structure to reflect the intended parent-child page relationships was not an option as the resource path links within the html files would be broken. We quickly realized that none of the existing Hierarchy Builder classes were going to work for our purposes and a new class would have to be created for this Mad Cap migration process. This new hierarchy builder class would customize the order in which the UWC tool uploaded the converted pages to Confluence to ensure that the page level hierarchy would be maintained after the migration. This was to be accomplished by referencing the XML file provided within the Mad Cap export data that provided the details of the hierarchical nature of the original documentation. On the initial upload to Confluence, pages within each hierarchical level were being ordered alphabetically by default, unfortunately there didn’t appear to be any way to control this on the initial page upload. To remedy this issue, we implemented a post page upload routine to ensure that the uploaded pages maintained their expected order within their hierarchical level. The Mad Cap Table of Contents XML file also provided us with the full name of each page as shown in the original Mad Cap Table of Contents which we were then able to use in order to provide Confluence with a friendly and unique page name going forward. This was also extremely helpful to us in converting internal page hyperlinks into Confluence markup since we could no longer rely on the file name and path specified within the original source output.

Linking Related Content Across Pages

One of the main features of any decent documentation or knowledge base enables the ability to link related pages and provide the user with the ability to quickly navigate to the identified related pages either directly or via keyword searching. Mad Cap provided users with the ability to navigate to related pages through a “See Also” link that when clicked, launched a popup generated through script that listed all of the pages that relate to the current page. Clicking on one of the linked topics (See screenshot below) would direct the user to the related page. We wanted to ensure that the related pages within the migrated documentation would be more obvious after the migration than what the end user experience would have been previously with the Mad Cap functionality. After some analysis of the macros built into Confluence that work with Page labeling, we felt that the Content by Label Macro fit the bill perfectly. This macro lists all of the pages that exist within the Confluence space that have a defined set of labels attached to the page. This would also have the added benefit of dynamically referencing any new pages added to the documentation in the future.

Content Linking in Mad Cap	Content Linking in Confluence

In order to accomplish this functionality, we created a new UWC Converter that would effectively analyze each html source page to extract the related concept information and automatically create page labels based on the concept relationships. These labels would then need to be fed into the Content by Label Macro and inserted at the bottom of each migrated page output in the expected Confluence style markup for the macro. Adding the extracted labels to the final converted Confluence page in addition to using them within the macro would allow for fully indexed searching of all of the resulting pages using the default search capabilities built into Confluence.

Expandable Content Containers

Many of the Mad Cap documentation source pages heavily utilized expandable containers. Within Mad Cap, these expandable containers were always initially closed upon navigation to the page, this allowed for a lot of content to be made available on each page while only initially displaying a subset of the data. We felt that this really reduced the clutter of content on each individual page and we needed to ensure that we maintained this behavior when migrating the pages into Confluence. After reviewing the available Confluence built-in macros we quickly found the Expand Macro to be a perfect fit for this functionality.

Expandable Containers in Mad Cap	Expandable Containers in Confluence

The automation of migrating the Mad Cap drop down containers into Confluence Expand Macros was a fairly straight-forward process since we could specifically identify these containers through a series of regular expression statements and could then perform the find and replace on every instance encountered using the UWC Java-RegEx class converter. Once we identified the required regular expressions necessary to match the entire container and specifically capturing the internal data that we wanted the Expand Macro to contain, the only work necessary was to add a line item into the configuration file for the UWC to call on during the page conversion process. No new converter class was required to be created to accomplish this portion of the migration, although we could have easily created one to do the work if any additional changes needed to be made to the macro that a straight forward find-replace routine couldn’t accomplish.

Highlighted Information Panels

One of the more prominent visual indications of important information within the Mad Cap source documentation were various highlighted text paragraphs that we needed to ensure would have the same level of visual importance after migration into Confluence. These visual indications were heavily used in the client’s documentation that we were migrating. The Panel Macro was the perfect fit to allow us to match the visual element almost exactly (and in our opinion, better).

Highlighted Information in Mad Cap	Highlighted Information in Confluence

A new UWC Converter class was created to identify the various ways in which these highlighted information panels were created in the Mad Cap source documentation. We were able to distinguish the panel types by their differing paragraph class attributes in order to style them with the appropriate colors in the resulting migrated Confluence page output.

The End Result Is What Matters The Most… 

In the end, our enhancements to the UWC tool allowed us to get approximately 99% of the conversion done as we expected in an automated fashion but because of the unstructured nature of HTML it was inevitable that there would be content that would still need to be adjusted manually. We were able to very effectively automate the migration process from Mad Cap Flare to Atlassian Confluence within the budget afforded to us, keeping in mind the high quality that our clients expect when working with Avant. We could have spent much more time refining and tweaking the tool and its associated converters to get that much closer to 100% but couldn’t justify the time and cost of doing so. With that in mind, a thorough quality assurance pass and tweak of each of the migrated pages within Confluence allowed us to have 100% confidence in the deliverable that we provided to our client to ensure they would be as happy with the resulting migration as we were!