Integrating your website with back-end systems has many benefits. These advantages range from improved data integrity across the organization, to real-time reporting, and greater employee optimization from removing manual data entry processes. Executing an integration can be a frustrating and expensive task if not planned correctly. This post will help guide you through the planning process by discussing five key areas to identify up-front that will save you time and headaches down the road.
Which Systems Should You Integrate and Why?
The first step is to define which system(s) you need to
integrate with your website to get the results that you want. Most integration projects start with a need to automate some business processes for one reason or another. Some examples include automating online order fulfillment or entering financial data into accounting software. Once you know the problem you need to solve you can identify the systems that website must communicate with to achieve the required solution.
How will the Systems Communicate?
Now that you know which system(s) the website must talk to, you need to decide how they are going to communicate with each other. This decision will depend on different factors such as what capabilities the back-end software has, how you want the software to process the data coming into it, what the requirements for data transmission are and more.
APIs
One option for communication is to leverage your software's
API through a web service. Most business software today has this capability, but you should verify that before choosing to go this route. A well-written API is ideal for integration because it typically means the software company has already written the logic to validate and process the data in their system. It is also is easier for developers to work with than some of the other means of integration. The APIs also come with developer support from the software company.
Database
Another option for moving the data from the website to your business system is by writing directly to the
database. You can either write straight into the production tables for the software or more commonly write to staging tables which the software then processes and imports into the production tables. The second approach is the safer one for obvious reasons. Using this method of communication will require the web server to be able to communicate securely with the database server for the software. We will talk about secure data transfer in a minute.
Flat Files
If API or database communication is not possible or not desired for some reason, another avenue available for integration is to transmit and process
flat files. This method is the most cumbersome of the options as it requires additional steps and layers of processing logic. We typically see this type of communication with older systems that don't provide an API or when direct access to the database is not possible for one reason or another. With this method, the website would generate a flat file such as a CSV file and upload it to a server. The software system or a custom business application is listening for these files and then processes them into the system.
What Data is Being Shared?
Determining who is going to communicate and how they will communicate are necessary steps to deciding how best to integrate your systems. You have to decide which data to share between the systems. Figuring out which data fields will go from the website to the back-end system is not the only part of this step in the planning process. You need to identify how the data from the website maps to fields in the back-end software. For instance, the website has a field for OrderTotal, which field in the accounting system does this match? Is it even a single field or does the software require this field to be created through a calculation of all the other line items in the order? These are important pieces of the planning phase that have to be mapped out before beginning any integration work.
In addition to mapping the fields between systems, you also want to compare the data types of the fields. Are the data types exactly the same or does some form of conversion need to happen during the integration? If a conversion is necessary, what is the proper way to convert the data without losing accuracy?
How do you Keep the Data Secure?
When sending data from one system to another, you will want to ensure you are doing it securely. This
security is especially important with financial and other types of sensitive data. Depending on the method of communication you choose, your options here will be different.
If using your software's API, you will want to ensure the web service endpoints have protection by SSL; this means they should be using HTTPS instead of HTTP. SSL will ensure the data is encrypted during transmission making it much more difficult for the data to be intercepted. Not all SSL certificates are the same, and encryption technology is continually being updated. You should check with your software provider for the exact details on how to protect communication through their API.
If you are achieving integration through direct database access or another means you must first determine how the web server will reach the destination. These typically live on different networks and the most common way of securing the communication is through a secure VPN connection. By connection through the VPN, the systems are treated as if they are on the same network, and therefore you can manage the security as you would on a standard network. The VPN connection often called a VPN tunnel, is secured using SSL or other encryption technologies – again making it very difficult for data theft.
What Happens When Communication Goes Down?
You now have the basics on how to plan the integration between your systems but the problem is that this assumes communication will always be up and running. However, that is not realistic – as much as we would all love to think that it is. The reality is that connections will go down from time-to-time, some will be less than a second and others will be much longer. You need to plan for this possibility and how to prevent data loss when it occurs.
If the connection between the website and your back-end system goes down, your website is not going to stop taking orders or collecting the data to send to the other system. Instead, it is going to try and send the data which will also send an error. That error will be logged, but that is where the communication will end. The transmission failed, and there is no retry logic in place, so everything just continues. Meaning that during those outages, the data is not making it to the other system thus causing the systems to be out of sync, and this is one of the reasons you are building the integration, in the first place – to keep things in sync. The best solution for this is to build in some logic with your integration that identifies transmission failures and allows the data to be retransmitted once the connection is back up and running.
Now You Have a Plan
Integrations can provide significant value to organizations and be a key piece of helping a business grow. Without proper planning though you can quickly run into many headaches and cost overruns. By reading this post, you should now have an understanding of how to create a solid integration plan to build out your website integration successfully.