Automated Accessibility Testing – Enabling accessibe product delivery to clients without discrimination

Ensuring accessibility and usability for aged & specially abled people is always a bottleneck, Over seven million people or 18% of the working-age population in Britain are disabled as defined by the Equality Act 2010. 

Web accessibility is a yardstick to measure application’s accessibility for the end users with disabilities
or impairments aside normal users. To support and comply with the ‘Accessible Certified’ web application to WCAG standards, the Quality Engineering team has adopted the SiteSpeed.io/ Lighthouse/ Axe-core packages.

Problem definition

Web sites should be accessible in order to provide equal access and equal opportunity to people with
diverse abilities

•  RNIB Concerns – RNIB discovers huge usability & accessibility issues during the acceptance testing phase. 90% of which are deviations from WCAG guidelines.
• Costly Defect Fix – Cost of fixing many defects identified during Acceptance testing phase is high
• Validating Accessibility Manually – Achieving WCAG rules for accessibility compliance for websites
  manually can be a nightmare. Validating accessibility for 100+ pages and web modals of a customer facing eCommerce website is a huge effort taking activity. It’s expensive too
• Quality & Eficiency – Loss of business due to delivering a website to disabled customers without
  thorough accessibility testing.
• Discriminative – Websites not accessible are discriminating and do not adhere to UK equality Act
  2010 as Over seven million people are disabled.

Key Solution Description

Websites of one of the Retailer’s in the UK are developed using React.js, the components of any given
page are developed and stored in Storybook where the QE team pairs with Developers and incorporates Axe
which validates the Accessibility in component level itself. This helps in reducing the basic issues identified by
RNIB during audits.

Sitespeed.io + Lighthouse + Axe is used to run through all 100+ web pages to identify the accessibility
violations vulnerabilities.

Page wise accessibility score Dashboard is generated based on the accessibility test execution which
enables the project team to keep track of pages getting improved or pages declining in accessibility score.

Along with Sitespeed.io + Lighthouse + Axe, QE team use Codecept.js + Puppeteer.js + Axe-core to
stimulate any given state in a web page to validate the accessibility in Popups and Modals. Huge manual effort
is avoided because of using Sitespeed.io, lightHouse, Axe, Axe-core.

These accessibility tests are executed in CI/CD Feature Jenkins pipeline before even your feature or
branch code is merged to Master. Our CI/CD Path to Live Jenkins pipeline triggers Automated Accessibility
test, where the Live build fails if Accessibility tests are failing. i.e The huge importance of Accessibility testing
is achieved before any feature is released to Production i.e to the end customer. Cost to fix defects during later
stages are highly avoided because of CI/CD runs.

Key Benefits

As our websites are developed using React.js, the components of any given page are developed and
stored in Storybook where the QE team pairs with Developers and incorporates Axe which validates the
Accessibility in component level itself.

• 0 Licence cost
• Solution is compatible with any Cloud based Software development
• Seamless CI/CD integration with Scalability feature
• Early issue identification
• No Manual effort required
• 95% of the issues identified during Automation Audits (i.e RNIB) are identified during
  Development phase itself
• Every Accessibility issue is segregated as Serious/ Complex/ Medium/ Minor categories to
  make BA’s work easy to prioritise the issues to be fixed earlier.

“Out of all the People Discrimination is purely avoided because of any physical challenges.” 

Cost Saved 

Automated Visual Testing – Adobe Experience Manager Author and Front End – Let’s save time

Usually a manual QA team has been the only way to get insight into visual UI changes, but it’s time- consuming and error-prone. With Automated visual testing, teams can deliver updates on time,and keep up without sacrificing quality.The automation of visual testing is useful in Modern-day delivery, both for quality and velocity. As a consequence of which we need to make sure every possibility of automating a process should be attempted for.

Adobe Experience Manager

AEM (Adobe Experience Manager) is a content Management tool , provided by Adobe . AEM is beingused by many companies because of its interactive capabilities and features. So basically, if a normal personwithout much technical knowledge wants to manage a website content (Texts/videos/Images/Digital assets) inan interactive way, AEM could be a choice for them.

Visual Testing

Visual Testing AKA UI testing verifies software/application from a purely visual Standpoint. It tests everything end-users see and interact with. And the QA team needs to make sure it looks the same as the UI developer’s specification. Visual tests generate, analyse,  and compare browser snapshots to detect for any UI level changes. These changes from the original UI design specs are called visual diffs.

Need for Visual Testing in Front End pages of a UK Retailer eCommerce website

On any website, there are many dynamic pages and many static pages too. Static pages don’t change for a long time, but how about Dynamic pages which get updated/modified every day or every week. In this case,we need to do regression in the Front end for every release we do to make sure no page is breaking, or there are no UI changes.

Need for Visual Testing in AEM

AEM has a UI too, which is called Author UI. Those UI comes from AEMSPA bundles. Nowadays Sites are built using common SPA frame works such as React. And we can point to the react end point in AEM to get AEM author UI – to make the experience interactive and beautiful. So basically, if we make any changes to FE react code, it will Break AEMUI too. And this can cause blocker for authors/Customers who use AEM within that organisation. And on average every organisation does 10-15 Frontend releases every week,which basically means doing regression more than 15 times every week which takes both TIMEANDMONEY. What if we automate this? SO, every time a team goes for release to production, those Visual testing will be done Automatically through CI/CD pipeline and notifies us if it breaks? And luckily, we were able to IMPLEMENT THIS within Organisation

Implementation

In the current market, we have many third-party tools which can be used to proceed with visual regression testing. But the question here is-How can we implement them in a project to make It useful and productive. Knowing tools is very easy, but implementing them is difficult, but luckily, we were able to use a third-party tool – Percy  within the organisation because of its good features and integration features. Tests are integrated with CI/CD-Jenkins which makes it more independent. Also it’s Cross-browser testing feature , Page and component testing , Responsive testing. Automating FE and AEM visual Testing has not only saved efforts but has also saved money, as it required less manual intervention. Which ultimately means less Human interaction, which means company Saving in terms of money Because it is Said: TIME IS MONEY and we saved time.

Cloud Testing

Essentially as more and more applications are being developed and hosted on cloud platforms like AWS, GCP, Azure, Digital Ocean and many more testing around these applications and their infrastructure also becomes essentially more complicated. Where cloud platforms optimise application deployments and performance as well as sunsetting of these applications with ease, the need for testing the functionality on these platforms is the key task 

We have made an effort to effectively test application hosted in cloud and its integration with AWS
managed services.

Problem Definition:

How does our ecosystem look?

Cloud Testing is testing activities around applications hosted on cloud platform which unlike on premise
applications have features like graceful scale up and scale down of application pods and relies heavily on containerisation platforms like docker or podman and orchestration environment like kubernetes or docker swarm or serverless platforms like AWS Lambda and adheres to tenets of Dev-Ops.

Example A REST API microservice (irrespective of its technology stack) needs external resources like Database hosted in the cloud often called RDS and CDK to have support so that applications can talk to this essential piece of infrastructure.

Challenges of testing in the Cloud?

Cloud environment comes with not just advantages but its own set of challenges and one of the key aspects are permissions and roles which decides what operations are allowed by applications to perform on AWS managed resources. Often such issues like applications not having permission to call databases (hosted on cloud) or publish a message into a queue(SQS queues on AWS platform) form a bottleneck to faster delivery and can be identified only in an integrated environment. But can we automate such scenarios?

How do you manage the cloud platform?

Cloud testing is not just about applications testing it includes an AWS platform with fine grained access control to all AWS managed resources to an individual in different environments which reflects our test beds where our applications are deployed. Each environment has a different goal to assess applications. While some focus on functional aspects other on non functional areas like security and performance including AWS configuration with these applications.

Example: Assessing an application just on functionality is not enough, cloudformation template for that
application must be assessed and all AWS resources which are also created as part of that application stack. Like a service with its necessary resources like RDS, SQS, SNS or S3 etc.

Key Solution Description:

What comes handy in this ecosystem?

Usage of cloud development kit, CDK is an open source software development framework for creating and defining cloud resources which hang together to give your application a structure through which it operates and executes its intended task. 

CDK library under cloud platform provides Test Constructs which comes with a bunch of fine grained assertions and test utils around cloud infrastructure. For example a service deployed on a cloud platform has also created associated SQS or SNS (asynchronous message processing infrastructure in AWS) when deploying the template.

Advantages of using Cloud Development Kit for automation?

The main advantage of testing applications in the cloud is they support a variety of DSL’s. We have used Java here, however the CDK library supports a wide range of languages like python, Go, Typescript, javascript etc. It also does not require that both development code and test code be in the same DSL and can be implemented based on individual’s choice.

Example: Dev code can be written in Java and Test code can be written in Python, however this is just for
effectiveness of CDK but in actual Industry is moving away from it and development code and test code shares the same technical stack and resides in the same repository.

CDK also provides an effective library to call on all AWS resources which can be orchestrated for test purposes in any environment and any DSL.

Key Benefits:

• If effectively utilized identifies majority of issues in initial round of functional testing
• AWS CDK comes free and does not incur any license cost
• Provides effective way of invoking AWS components programmatically rather than completely verifying manually
• Most of the tests which one performs in the AWS console can be automated without opening the console at all
• All tests run as part of the pipeline in various environments validating functionality is intact
• Reduces manual effort on validating tests around AWS managed resources