All of the supermarket giants allow a select few suppliers to provide an external website to supply goods and services, linked from the main website of the retailer. My customer, a web design and development agency, needed a specialist to build an ecommerce storefront that would integrate with a particular supermarket giant’s shopping cart system, both for sending customers to their website to take payment, and then to receive and process orders.
My task: build the online shop using their designs, integrate it with a 3D personalisation system, integrate it with the supermarket’s ecommerce shopping cart, allow orders to be received electronically from the supermarket, and then dispatch those orders electronically to multiple suppliers. Simple!
The web development company called on me to work closely with their technical director to take the shop right through to launch, and then provide second-line support and maintenance afterwards. In this article I’ll call the supermarket giant “BigCo”.
Linking with the Supermarket Giant’s Shopping Cart
I created an add-on for the “Magento” e-commerce storefront (a “Magento extension”), which was able to bypass Magento’s built-in checkout and payment system, and instead use a method called “SOAP” to add items to, and link back to, BigCo’s own ecommerce cart. When the user decides to finish shopping on the site, they are able to click through to BigCo’s shopping cart and complete the payment process.
An important requirement of this was to ‘hook’ into the Magento system in a way that did not interfere with its normal operation. This required a wide spectrum of knowledge – from Magento’s core programming code, through to understanding how customers and staff would actually use the system.
This ‘seamless integration’ (in other words, keeping the system as ‘normal’ as possible) saves money by allowing a web designer to work on the look and feel of the website, using all of their normal skills, without having to understand how this ‘behind the scenes integration’ actually works.
The main technology involved is SOAP or “web service” requests, triggered by the user’s actions, sending messages to BigCo such as “add this item to the cart” or “tell me what’s in the cart”. Each user has a “session” (a way of storing temporary information about website users during their visit), and part of my code had to ensure that the Magento session information keeps track of the BigCo session information, so that both ends of the system are kept in sync.
Receiving Orders via XML into Magento
When an order is paid for, BigCo’s system will upload an “XML” file containing all the order details, to the recipient’s web server. The website needed to regularly check for new orders, then process them reliably, and add then to the Magento system, and then send BigCo an acknowledgement. At a later date, when the Magento system is told that the order has been shipped, it then had to notify BigCo’s system in turn.
Managing a Large Set of Complex Products
The term ‘complex products’ means different things to different people. In Magento terminology, it refers to products that have a several variations, each with their own product code (or ‘SKU’). I’m expanding that term today to include products which are a single product code, but which have various configurable options (‘custom options’ in Magento terms).
Typically the team will go through several stages of product management for a new shop:
- Initial testing or sales demo: a small set of products are manually created on the store
- Product information arrives from suppliers during store development: a spreadsheet is created as a ‘quick solution’ and is used to upload a large batch of product data. This may take several attempts before the format is right.(at this point, it’s vital that you decide how product data will be managed long term – see below)
- The store is nearly ready, at the testing/tweaking stage: the product data is probably still in spreadsheet form somewhere, which makes it easy to perform large-scale changes (eg. re-categorising 300 products), but you’ll probably be getting a stream of data corrections back from the testers on all sides, and it’ll be tempting to manually tweak the product data on the store to make these corrections.
- The store is live: at this point, you need to have agreed a consistent way to manage product data with everyone involved, which will allow all anticipated changes to be handled efficiently, and is easy enough for non-technical admin staff to do.
- Manually add/update on the store itself. The advantage is that individual changes can be made quickly and easily by non-technical staff, but it becomes quite difficult to make large-scale changes (eg. search and replace terms in the product descriptions, or increase all prices by a set percentage). This doesn’t count if you’re using the Magento system as your ‘master’ store of product and stock information – see below.
- Spreadsheet: keep a ‘master spreadsheet’ of the product data, nominate someone to be the curator of it, and ensure that they know how to import it into the site and any constraints (eg. product names can only be X characters long). The advantage is that all the product information is accessible in a flexible format, and large-scale changes can be implemented easily. The disadvantage is that if you accidentally make a large-scale change (eg. deleting a column or using an out of date sheet), and then import that data into the website, it can cause problems across the entire site.
- Separate System: if you have an internal system for managing product data, perhaps for controlling stock levels and handling telephone orders, and all staff use this as the ‘master’ for product related work, then you should look at automating the link between this system and your Magento website. I have project managed and developed several such website integration/automation systems in the past, and although it takes some time to ensure all requirements are met, it allows businesses to grow quickly without the website costs spiralling out of control.
Integrating with a large retailer like this has its fair share of challenges. The most notable challenges are:
- understanding documentation which, due to the rapid pace of changes in the retail world, is often a very fluid document. To understand this successfully, it is important to form a good relationship with the project managers and technical team at all the companies involved in the integration, to understand the various pressures they are under from other areas of the business, and to “speak their language” whether that be commercial, or even a specific programming language which may not be the one you’re actually using.
- being flexible: software developers are often creatures of habit and conformity; we love to have order and predictability. However, when integrating systems using knowledge that may or may not be accurate from people who may or may not fully understand the issues mandates having somebody who can forgive documentation errors, who can use years of experience to guess at several likely corrections, and who can say the right things to the right people to home in on the most efficient solution without causing stressful misunderstandings.
- understanding costs: projects involving more than one technical team often means having spiralling costs due to problems that were not predicted and not budgeted for. Having a technical person who can grasp the commercial needs of the project and therefore push for the right solution to a given problem can save money that would have otherwise been spent on unnecessary software development or procedure changes.
- wide ranging experience: many junior software developers, when faced with an unknown 3rd-party software system, will struggle to understand how that system might work internally, and this often shows up when data transmitted from one system to another is mis-interpreted (is that price in pounds or dollars?), incorrectly encoded (have you ever seen the text “&” instead of a simple “&” in a printed document?), or worse, has information missing (eg. all the line breaks missing and paragraphs running together)