Functional testing is a software testing process used within software development in which software is tested to ensure that it conforms with all requirements.
The concept
The idea of a functional test is to make sure that nothing visible to the customer broke. Functional tests don't care about how something is built they are more about how something works at a high level. I think they are also a good starting point for any refactoring as they can ensure that everything works as it was.
Magento 2 Implementation
In Magento 2 functional tests are done using Selenium simulating the behaviour of a user visiting our website. For running Magento 2 functional tests we need to install the Magento Testing Framework (MTF) and Selenium first:
- cd [M2_ROOT]/dev/tests/functional/
- composer install
- Set the configuration in [M2_ROOT]/dev/tests/functional/phpunit.xml
- Set the configuration in [M2_ROOT]/dev/tests/functional/etc/config.xml
- Set the configuration in [M2_ROOT]/dev/tests/functional/credentials.xml
- Download the latest version from http://www.seleniumhq.org/download/ (Selenium Standalone Server)
- Start the selenium server
bash java -jar [path_to_selenium_directory]/selenium-server.jar
You can find more detailed instructions in the Magento 2 DevDocs here and for the Configuration here.
Example
We are going to create a small test for a simple feature and run it.
The goal is to add our own block with template on the catalog product view page. We are then going to test if we did it correctly with a functional test.
Building our feature
Let's add the block first by creating a layout that is named after the handle:
<?xml version="1.0"?>
<!-- src/view/frontend/layout/catalog_product_view.xml -->
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
<body>
<referenceContainer name="content">
<block class="Magento\Framework\View\Element\Template" name="fooman-example-block" template="Fooman_BlogExample::example.phtml" />
</referenceContainer>
</body>
</page><!-- src/view/frontend/templates/example.phtml -->
<p class="fooman-blog-example">Hello world</p>Functional Test
We are now going to create a functional test to verify that our feature is working as expected:
The concept
In this test we just need to hit our catalog product view page and then try to find the html that we have added in the response. There is more than one way to create a test, we just picked an approach that seems simple enough to get started.
Our test will have four parts:
- Test Case
- Page
- Block
- Constraint
1. TestCase
A test case is the base for our test. In it we can set some initial state for our test and define different variations. An MTF test case usually consists of an xml file and a php file.
In our case the goal is to create a state that is needed for our test. We do so by injecting pages and fixtures into our test class and doing certain operations that are needed.
A page represents a single class that holds the response of some HTTP request.
We need admin catalog product index page and admin catalog product new page so that we can create the product that we are going to check on the frontend.
Fixture factory is what we use to create objects with a given dataset.
<?php
// tests/functional/tests/app/Fooman/BlogExample/Test/TestCase/CatalogProductViewExampleTest.php
namespace Fooman\BlogExample\Test\TestCase;
use Magento\Catalog\Test\Page\Adminhtml\CatalogProductIndex;
use Magento\Catalog\Test\Page\Adminhtml\CatalogProductNew;
use Magento\Mtf\Fixture\FixtureFactory;
use Magento\Mtf\TestCase\Injectable;
class CatalogProductViewExampleTest extends Injectable
{
/**
...
*/
public function __inject(
CatalogProductIndex $catalogProductIndex,
CatalogProductNew $catalogProductNew,
FixtureFactory $fixtureFactory
) {
$this->catalogProductIndex = $catalogProductIndex;
$this->catalogProductNew = $catalogProductNew;
$this->fixtureFactory = $fixtureFactory;
}
/**
...
*/
public function test($product)
{
list($fixture, $dataset) = explode('::', $product);
$product = $this->fixtureFactory->createByCode($fixture, ['dataset' => $dataset]);
$this->catalogProductIndex->open();
$this->catalogProductIndex->getGridPageActionBlock()->addProduct();
$this->catalogProductNew->getProductForm()->fill($product);
$this->catalogProductNew->getFormPageActions()->save($product);
return ['product' => $product];
}
}Next let's look at our xml file.
For this test we need to, as the name suggests, define our test cases and define the variations that will be used if needed. Variations are predefined structures that can contain:
- Sets of data that are relevant for the test variation
- Constraint class for the particular test variation that decides if our test fails or passes
Since we want to keep our example simple we'll just have one test case with one variation and one constraint using one set of product data:
<!-- tests/functional/tests/app/Fooman/BlogExample/Test/TestCase/CatalogProductViewExampleTest.xml -->
<?xml version="1.0" encoding="utf-8"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="../../../../../../vendor/magento/mtf/etc/variations.xsd">
<testCase name="Fooman\BlogExample\Test\TestCase\CatalogProductViewExampleTest"
summary="Test that the Example block is added" ticketId="401">
<variation name="ExampleBlockVariation1">
<data name="product" xsi:type="string">catalogProductSimple::product_10_dollar</data>
<constraint name="Fooman\BlogExample\Test\Constraint\AssertExampleBlockExists" />
</variation>
</testCase>
</config>We can reuse the catalogProductSimple::product_10_dollar fixture that is part of the standard Magento 2 Catalog package.
In our constraint we will just assert that the page contains the html that we added.
2. Page
As we mentioned, in functional tests a page represents a class around an HTML document that is returned as a result from an HTTP request. These pages are created in XML format and generated with a CLI command.
Pages contain blocks which are parts of a page.
The concept of blocks is different from standard Magento as in Magento 2 functional tests blocks are parts of the page. They are created with the selectors which can be css or xpath.
We need a catalog product view page and a block that is actually a div with "maincontent" id as that's where our part of the code is supposed to appear.
<!-- tests/functional/tests/app/Fooman/BlogExample/Test/Page/CatalogProductView.xml -->
<?xml version="1.0" encoding="utf-8"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="../../../../../../vendor/magento/mtf/etc/pages.xsd">
<page name="CatalogProductView" mca="catalog/product/view" module="Magento_Catalog">
<block name="foomanCatalogProductPageMainBlock" class="Fooman\BlogExample\Test\Block\Catalog\Product\View" locator="#maincontent" strategy="css selector" />
</page>
</config>3. Block
A block in MTF is a class that contains a part of some page. We defined our block in our page xml (everything inside div with "maincontent" id) and now we need to create a class for it.
In this block class we need to return the markup that is supposed to be there. We'll use a css selector for that:
<?php
// tests/functional/tests/app/Fooman/BlogExample/Test/Block/Catalog/Product/View.php
namespace Fooman\BlogExample\Test\Block\Catalog\Product;
use Magento\Mtf\Block\Block;
use Magento\Mtf\Client\Locator;
/**
* Class View
* Catalog product page view block
*/
class View extends Block
{
/**
* Check if blog example block exists
*
* @var string
*/
protected $blockSelector = '.fooman-blog-example';
/**
* @return string
*/
public function getBlogExampleBlockText()
{
return $this->_rootElement->find($this->blockSelector, Locator::SELECTOR_CSS)->getText();
}
}4. Constraint
The Constraint is a class which purpose is to run the assertion on the block. We need to fetch our block and check if the markup that we added exists.
We are first going to inject all the objects that we need for this test in the processAssert method:
- catalog product view page
- product object
- browser object
We will then get the block from our page and call the block method that we have previously created:
<?php
// tests/functional/tests/app/Fooman/BlogExample/Test/Constraint/AssertExampleBlockExists.php
namespace Fooman\BlogExample\Test\Constraint;
use Magento\Catalog\Test\Page\Product\CatalogProductView;
use Magento\Mtf\Constraint\AbstractConstraint;
use Magento\Catalog\Test\Fixture\CatalogProductSimple;
use Magento\Mtf\Client\BrowserInterface;
class AssertExampleBlockExists extends AbstractConstraint
{
/**
* Assert that our example block is added
*
* @param CatalogProductView $catalogProductView
* @param CatalogProductSimple $product
* @param BrowserInterface $browser
* @return void
*/
public function processAssert(
CatalogProductView $catalogProductView,
CatalogProductSimple $product,
BrowserInterface $browser
) {
$browser->open($_ENV['app_frontend_url'] . $product->getUrlKey() . '.html');
\PHPUnit_Framework_Assert::assertSame(
'Hello world',
$catalogProductView->getFoomanCatalogProductPageMainBlock()->getBlogExampleBlockText(),
'Block not found.'
);
}
}Running the test
One thing to note is that currently functional tests in Magento 2 don't support namespaces that are not Magento's. If we want to create our own module and run our own tests, as we do in this example, we will need to add our own autoloader path in the [M2_ROOT]/dev/tests/functional/composer.json file:
"Fooman\\": ["lib/Fooman/", "testsuites/Fooman", "generated/Fooman/", "tests/app/Fooman/"],
For this to take effect we need to run:
composer dumpautoload
Before we run the tests we also need to generate our page from the xml:
cd [M2_ROOT]/dev/tests/functional/utils
php generate.phpFinally let's run our test with:
cd [M2_ROOT]dev/tests/functional
phpunit --filter CatalogProductViewExampleTestConclusion
Functional tests can be very useful in Magento as they provide a way for us to quickly ensure that what is built works as expected. Their reusability provides us with different paths from which we can choose when modeling these tests.
We have only scratched the surface with our example but let us know in the comments how you get on.
The complete example can be found on github.