Test Automation: 3-Day Proof of Concept by Pragmasys Consulting

Microsoft Dynamics 365 is a really popular choice for businesses looking to manage sales, service, and other processes. With more and more people using it and Microsoft updating it regularly, it’s tough for organizations to quickly and reliably test the health of their systems after each update.

Test Automation 365 is designed to improve the quality and speed of testing by reducing manual effort. There are a few reasons why we need a Dynamics 365 test automation solution. For example, there are loads of regression tests that have to be done before each release, the business logic is really complicated and linked to the CRM interface, and we need the system to keep on going. There are various Dynamics 365 automation testing tools to make this easier, but it’s tricky to implement automation specifically under Dynamics 365. So, loads of companies are looking for practical ways to try out automation on their solutions without taking too many risks. This is where short pilot projects, or proof-of-concept projects as they’re also known, come in.

What Is the Pragmasys Consulting 3-Day Test Automation PoC?

Test Automation: 3-Day Proof of Concept is a short-term consulting solution from Pragmasys Consulting, a kind of dynamics automation test solution that allows you to implement basic test automation for Dynamics 365 in three days. The purpose of this test automation solution for Dynamics 365 is to demonstrate in practice how automation can increase testing productivity and optimize costs when maintaining a client’s CRM system. In fact, Pragmasys offers interactive cooperation: their experts, together with the customer’s team, select one business process in Dynamics 365 and, in 3 days, automate the testing of this process.

Such a pilot project serves several purposes. First, it clearly confirms the viability of the automation approach in the customer’s specific environment. Secondly, PoC helps to evaluate what results can be achieved – for example, faster regression tests, increased test coverage without increasing the testing staff, etc. Third, PoC results in real artifacts (scripts, frameworks, reports) that remain with the customer.

Key Features of the Pragmasys Test Automation PoC Solution

Pragmasys’ PoC is focused on building a working mini-version of the Dynamics 365 test automation system. The main features and components that this solution includes are listed below:

• Proprietary automation framework on Selenium: Experts here at Pragmasys create automated scripts based on Selenium WebDriver and C#. Dynamics 365 test automation tool Selenium is popular because it’s flexible. The scripts simulate user actions in the Dynamics 365 interface, performing tests just like a human would.
• Coverage of different types of tests: The created scripts can do different types of tests, like smoke or sanity tests after deployment, or regression runs of a full scenario. Pragmasys is all about test classification: you can set up suites in the PoC for quick runs of key features or more in-depth regression. Also, if it’s relevant to the project, the framework lets you automate data migration checks.
• Automated reporting and logging: As part of PoC, you can set up a detailed report on the results of test runs. The report includes stats on passed/failed cases, graphs and summary tables (Pivot), which visually reflect the quality of the build or upgrade. If a test crashes, the framework can take screenshots of the screen for either all steps or just the dropped asserts. These log files and screenshots should help you figure out what caused the failure quickly.
• Consideration of safety features and environment: The solution takes into account the specifics of corporate systems. In particular, support for multi-factor authentication (MFA) for access to Dynamics 365 is implemented. If the login to CRM is protected by MFA, scripts will be able to bypass this – so automated tests are not “broken” due to security requirements.

Daily Program of the 3-Day PoC Program by Pragmasys Consulting

But, let’s be real, it’s pretty much impossible to automate everything in just three days. But don’t worry, Pragmasys has come up with a great plan to get you useful results in no time at all. The 3-Day Proof of Concept is broken down into stages by days:

• Day 1: Analysis and Planning. On the first day, the team meets with the customer to get their requirements and understand the business process that’s been chosen for automation. The partners then clarify which functional points (up to 3) are included in the scenario, and which test cases already exist. First, initial complexity assessment. Then, Pragmasys gets things set up for testing at the client’s place. In fact, the POC project is set up and work planning for the rest of the days happens. By the end of the day, the automation plan for the selected process and the basic infrastructure should be ready.
• Day 2: Developing test scenarios. On day two, focus on creating automated tests. If the customer has already written up test cases for this process, Pragmasys will take those as a starting point and write scripts based on them. If there aren’t any ready cases, the consultants and the client’s business experts will put together the necessary tests from scratch, describing the steps manually, and immediately start automating them. Development is done in C# + Selenium: functions are written to navigate the Dynamics 365 form, input data, and validate results. Also write auxiliary scripts for things like data preparation or environment configuration. By the end of the second day, you should be able to cover all the agreed functionality (up to three key steps of the process) with scripts and conduct initial runs of individual parts of the script.
• Day 3: Running tests and demonstrating results. On the last day, Pragmasys runs the automated scripts that they’ve developed. Tests are run in batch, and then it’s just a case of checking they passed and fixing any remaining errors in the scripts. A detailed report on the runs is generated, showing the overall results and step-by-step details. If some tests fall down, the team looks at the logs and, if needed, tweaks the scripts to get things running smoothly. Then, the results are shown to the customer, and they’re told how the tests are run, how reports are generated, and where to find logs and screenshots.

Pros and Cons of the Pragmasys Dynamics 365 Test Automation Solution

As in any test automation solution review, let’s highlight the strengths and weaknesses of Pragmasys 3-Day PoC.

Pros of the Pragmasys PoC Solution:

• A fast and free start. The main advantage is the absence of direct expenses from the client’s side: Pragmasys offers to conduct a three-day PoC free of charge, which lowers the barrier to the introduction of new technology. In just a few days, the customer gets a tangible result, whereas a full-scale automation project would take weeks or months. Quick results help to interest stakeholders and get support for further development of the initiative.
• Expertise and quality of execution. Pragmasys is a long-standing Microsoft partner with top-notch skills in Dynamics 365. This means that PoC is carried out by pros who know the usual business processes and pitfalls of CRM testing. When it comes to PoC, we make sure to apply the best automation practices. This includes using proven tools like Selenium and C#, setting up test structures with smoke and regression sets, and managing test data and reporting in a competent way. The customer gets a mini-consulting session on how to set up automated testing, along with a ready-made example.
• Specific practical outcome. It’s not like theoretical evaluations or demonstrations on abstract scenarios, where the value is in the abstract. This PoC delivers value directly in the customer’s environment. It’s a real business process that’s important for the company, not just a training example. The stuff that comes out of the process, like test scripts, frameworks, and reports, stays with the client. They can then use it again or use it as a starting point for more work.
• Dynamics 365 specifics. The solution is customized for Dynamics 365 CRM, and there are a number of features that prove this: It supports MFA authentication, works with the cloud interface, and can generate data via API or UI, accounting for different modules (e.g. Sales, Service, etc.). This is what makes PoC different from using universal frameworks that don’t consider the specific features of the platform.

Cons of the Pragmasys PoC Solution:

• Limited Coverage. The most obvious disadvantage is the small coverage volume. One business process (up to 3 functional points) is a drop in the ocean for most systems. The rest of the Dynamics 365 scenarios remain untested. After PoC, the company still won’t have a full-fledged regression autotest suite – it needs to be developed separately. Thus, the value of PoC is of a demonstration nature: yes, it proves the feasibility, but it does not solve the entire testing problem.
• The need for further investment. Once the PoC is done, the client can decide to either carry on the automation in-house or get Pragmasys/another contractor to take on a full-fledged project. It’ll take time and money to make sure the rest of the functionality is as successful as PoC. The free pilot is just a way to test the water, and then there’s a paid phase to get everything up and running. If an organization isn’t willing to invest, the benefits of PoC will just remain unrealized in practice.
• Dependence on the expertise of an outside contractor. Code and tools that are created in 3 days are probably written in a style that the Pragmasys team will recognize. If we can’t work together or hire autotest specialists, it’ll be hard for the company to keep this framework going and improve it on its own.
• Possible technical limitations. I’d say that UI automation with Selenium is pretty powerful but also a bit fragile. In Dynamics 365, the user interface changes a lot (especially during major Wave updates), which can sometimes cause problems with scripts.

Common Use Cases for the Pragmasys 3-Day Dynamics 365 Test Automation PoC

Let’s consider which cases and companies such PoC is particularly suitable for. As a rule, a short test automation trial project is used in the following situations:

• Implementing regular updates. Companies that use Dynamics 365 and get regular updates want to make sure that these updates don’t mess up important business processes. For example, a bank or a retail company can choose the most important scenario and automate its validation.
• Transition from manual testing to automated testing. Organizations that do Dynamics 365 testing the old-fashioned way, with people spending ages doing it manually, are thinking about getting automated systems in place, but they haven’t really gotten their hands dirty with it yet. At the end of the day, PoC is just a way of testing the hypothesis: will we benefit from automation? So, by picking a great process, you can easily compare how much time is spent on running the script manually and on the autotest, how stable the script is, and what problems come up.
• Before/after a major CRM implementation. Some companies order PoC at the same time as or just after implementing Dynamics 365. For example, once the sales module has been set up, the customer can immediately run automated tests of key functions (like lead generation and transaction execution). This is to make sure the new system works as required and will continue to work with further changes. This approach is popular in businesses where there’s a lot of pressure (like finance and telecom) because if you make a mistake, there’s a high cost.
• Evaluating a specific tool or approach. Sometimes, PoC are asked to try out a particular technology. For example, a company might be thinking about using a particular automation platform. Pragmasys uses a classic stack, but if you want, the tests can be integrated with Azure DevOps, run in the cloud, etc. If a company wants to see how automated processes fit in with its CI/CD workflows, PoC can show them. There are other options too, like specialized Dynamics automation test apps from third-party developers.

Cost and Inclusions: What’s Inside the Pragmasys PoC Package?

As mentioned above, Pragmasys’ 3-Day PoC is offered at no cost to the customer. This is a typical model for consulting services on Microsoft AppSource – a short assessment or PoC for $0, which implies a further commercial phase. However, the freebie covers the entire scope of work for three days and all deliverables. The PoC package typically includes:

• Pragmasys specialist services within 3 days. The team consists of a Dynamics 365 consultant and an automation engineer (possibly one person with both competencies) to analyze, develop, and present the solution.
• A ready-made set of autotests for a single business process. When the PoC is finished, the client gets the source code of the test scripts (in C#), a customized project (framework) to run them, and documentation on its structure. The number of scripts depends on how complex the process is, but usually we use up to three (e.g. record creation, status update, and report generation – for example, in the sales process). All these tests are linked to the module you’ve chosen (Sales, Service or Marketing).
• Tools for data and results. The deliverables include auxiliary test data generation scripts, configuration files, and example test cases on which the automation was based.
• Presentation and Recommendations. As well as the technical stuff, Pragmasys usually gives the customer’s management team a report on the PoC that was done, like what was automated, what errors were found, how long it took to launch, and what needs to be done to expand coverage.

It’s important to note that even though the service is aimed at US customers, PoC isn’t limited to a specific geographical area. Pragmasys can work remotely, while Dynamics 365 itself is a cloud-based solution.

Comparison with Full-Scale Test Automation Services

When it comes to scope and duration, a full Dynamics 365 automation project can cover loads of business processes and hundreds of test cases and can take several months. It includes framework architectural design phases, CI/CD integration setup, lots of iterations of test development and stabilization, and staff training. A 3-Day PoC, on the other hand, fits into 72 hours and focuses on one narrow area. It’s more of a way to show what you can do, while a full project is putting all that into practice. For example, a full implementation might use various automation tools (like the Dynamics 365 test automation tool, in the broadest sense, including frameworks, libraries, cloud services), as well as continuous testing methods, integration with Azure DevOps to run tests at each build, etc. Of course, PoC doesn’t go that far.

So, the end result of using our full automation service is that we’ll have an autotest system that’s been integrated into your development and maintenance processes. It normally includes hundreds of scripts for all the main functions, as well as a test data management system and maybe integration simulation, and so on. This is an industrial solution that needs support. With PoC, you end up with a prototype, which you can develop into something bigger, but it’s not meant to be used in the industry for very long without changes. But a high-quality prototype from Pragmasys is pretty much a “combat” solution, as it’s got structured code, reports, and logs, which are a key part of full-fledged autotest systems. The difference is in the scale and degree of debugging.

When it comes to price, the difference is as clear. PoC is free of charge, but a large project is a tangible budget item. Test automation is an investment that doesn’t pay off straightaway. Lots of consultants offer PoCs or short assessments just to convince the customer to spend more. Usually, the cost of full services is calculated using the man-months of automators’ work and the cost of tool licenses (if paid platforms are involved). Just so you know, there are also commercial platforms out there for testing Dynamics 365. It’s a kind of app dynamics tool, and it can be used to create solutions based on Record & Play or specialized add-ons. It’s a great option if you’re looking to save some time. But Pragmasys shows how it’s done without buying ready-made software. It uses an open-source tool (Selenium) and custom development. So, the company reckons it’s possible to do without spending a fortune. On the other hand, ready-made Dynamics Automation Test Apps from third-party vendors can offer broader capabilities straight away.

Conclusion

So, our test automation dynamics solution review: Pragmasys Consulting’s 3-Day Proof of Concept is a great way to quickly see how test automation in Dynamics 365 can benefit your business. When the digital CRM platform is regularly updated and customized to meet the business’s needs, automated tests become a key part of keeping things stable and saving resources.

The best things about the offer are that it’s free, the people doing it are really good at what they do, and it’s all about getting real results. This makes it a great choice for companies just starting out on their automation journey. Pragmasys shows that even a small 365 test automation solution can get real productivity gains and cut costs on long manual test cycles.

Obviously, the pilot has its limitations. The functionality coverage is minimal, and we’ll need to put in some extra effort to achieve full coverage. But, at the end of the day, that’s what PoC is all about – working out what’s needed to keep things going. If you can get a clear example, you can make an informed decision about scaling automation. For example, you can decide whether to engage a partner for the full project or consider third-party tools. Either way, 3-Day PoC reduces uncertainty and provides valuable experience.

Oleksandr Volkovitskiy

Oleksandr is a passionate technical professional with extensive expertise in Microsoft Dynamics 365 Customer Engagement (D365 CE) customization, Canvas App development, and Power Automate solutions. His skill set includes Azure Function and Azure Durable Function development, Logic Apps, and seamless integration of PCF components into Canvas Apps. With a deep understanding of Azure technologies and a strong background in developing tailored solutions for D365, Oleksandr brings a comprehensive and forward-thinking approach to enterprise application development. He enjoys exploring new Microsoft solutions and shares his insights with the readers.