The results and lessons learned from eight years of chain tests for NS

Detect errors at an earlier stage by testing in the chain during the development phase

In 2012 NS organised a mini-competition for a chain test cycle. InTraffic won the competition and involved ACC ICT as its partner in the domain of infrastructure. This collaboration is coming to an end after a period of eight years because the chains tests have been made part of an NS-wide Request for Tender for a cloud platform. NS, InTraffic and ACC ICT are looking back at a successful collaboration. “The chain test cycles allow us to test the infrastructure as early as the development phase. This infrastructure reacts in the same way as our production environment, and this enables us to detect and resolve errors at a much earlier stage. This has reduced the length of our development cycle, improved our time to market and lowered the costs”, says Onno Wierbos, non-functional test lead in the NS Test Competence Centre.

In 2012 the IT organisation of NS was fragmented: a scattered application landscape as well as a scattered data centre landscape. The organisation was comprised of some twenty data centre locations, and a number of applications were hosted from a supplier’s facilities. As a result, it was virtually impossible to test chains of applications while the number of interfaces between applications was increasing continuously. This is the reason why NS urgently needed virtualised test environments in which they could perform automated chain tests. Initially this involved the travel information via NS.nl, the NS travel planner and OBIS, the On Board Information System in trains, which allowed travellers to check whether a train was moving according to schedule, and if not, the length of the delay.

NS organised a mini-competition for this project, which was won by InTraffic in close collaboration with ACC ICT. InTraffic carried out the project’s software component including the test automation, while ACC ICT was responsible for the infrastructure and hosting. NS came up with the name for the new chain test cycle: KWARTS, which is an acronym for Chain (Keten) Guarantee (Waarborgen) as Regular Test Service.

Ketentesten NS InTraffic

Familiar with organisation and applications

KWARTS is a pre-acceptance environment that allows you to test the effects on the chain on an ad hoc basis. Wierbos: “The old environment had the drawback that it took a long time to receive a time slot allocation if you wanted to reserve an acceptance environment. Moreover, this was expensive. KWARTS removed those limitations. Each developer can now perform a chain test at any moment in time and simulate the desired situation, from latency to performance.”

The old environment was built and completed eight years ago and has had an availability level of nearly 100% (99.9995%). Wierbos calls it an outstanding achievement. “The systems always worked well because InTraffic is thoroughly familiar with our organisation. They know the applications we use, the interfacing between these applications as well as the hardware requirements imposed by the applications. They also understand the structure of our business processes as well as the role played by the applications. As a result they have an in-depth understanding and knowledge of our requirements for properly performing chain tests.”

“We can detect and resolve errors at a much earlier stage, which allows us to accelerate the development cycle and reduce costs”ing up with ICT Group we’ve effectively built a future proof platform.”

Two areas of expertise that function as a single unit

The collaboration with ACC ICT proceeded seamlessly during the entire period. “NS had a contract with InTraffic, and not with ACC ICT, but at any moment we also communicated directly with ACC ICT. We operated as a trinity”, says Wierbos. What he found remarkable and laudable was the fact that any issues were resolved before InTraffic and ACC ICT would investigate who was responsible. “This was extremely pleasant for NS, for it meant that we had procured two different areas of expertise while they functioned as a single unit.” An important contribution to the success was the stable team composition, both from the side of InTraffic and ACC ICT. Wierbos: “There was no annual jostle of positions. And in those cases in which a change of the team did occur, the quality of service remained at its usual, high level because there was sufficient experienced staff involved in the project.”

Flexibility and speed in DTAP process

KWARTS has enabled NS to perform tests as early as the development phase in an environment which is an excellent representation of the production environment. This ensures a high level of flexibility and speed in the DTAP process. “We can detect and resolve errors at a much earlier stage, which allows us to accelerate the development cycle and reduce costs. We have managed to reduce the cycle from one to two weeks to a timespan of two hours within the LTSO domain”, says Wierbos. Another area of cost savings for NS is the speed with which they can switch between the design and the realisation of the further development of the environment. “The size of the environment has increased substantially over the years, and it became larger and wider. Thanks to the fact that InTraffic and ACC ICT were so thoroughly familiar with our requirements, NS has been able to realise this in an extremely cost-effective manner”, Wierbos adds.

“KWARTS has enabled NS to perform tests as early as the development phase in an environment which is an excellent representation of the production environment. This ensures a high level of flexibility and speed in the DTAP process“

Successful cooperation

Although in actual fact ACC ICT acted as the Hosting & Connectivity department of the NS Test Competence Centre, formally the company was ‘merely’ a subcontractor. “But we never played the game in this manner, and neither did NS or InTraffic”, says Paul Bijleveld, ACC ICT’s director. “We always had direct contact with NS at an operational level. All extra work and any occurrence of escalation was always discussed fully and openly between the three parties. We always make sure that the customer’s interest comes first. It is my belief that this is the most important success factor.” Wierbos concludes: “When I look back on the project, what stands out most is the way in which we worked together. The way in which InTraffic and ACC ICT have been working together for eight years is a prime example of the way in which the cooperation between customer and supplier should be set up. It clearly shows how the contract is secondary to the mindset of suppliers.”

Cookies

The ICT website uses cookies for several functionalities within the website, such as functional cookies and website statistics.

Read our privacy policy here | Close this notification
Scroll to Top