Welcome to Pytest-BDD-NextGeneration’s documentation!¶
BDD library for the pytest runner¶
pytest-bdd-ng combine descriptive clarity of Gherkin language with power and fullness of pytest infrastructure. It enables unifying unit and functional tests, reduces the burden of continuous integration server configuration and allows the reuse of test setups.
Pytest fixtures written for unit tests can be reused for setup and actions mentioned in feature steps with dependency injection. This allows a true BDD just-enough specification of the requirements without obligatory maintaining any context object containing the side effects of Gherkin imperative declarations.
Note
Project documentation: pytest-bdd-ng
Install pytest-bdd-ng¶
pip install pytest-bdd-ng
Project layout¶
pytest-bdd-ng automatically collect *.feature files from pytest tests directory. Important to remember, that feature files are used by other team members as live documentation, so it’s not a very good idea to mix documentation and test code.
The more features and scenarios you have, the more important becomes the question about their organization. So recommended way is to organize your feature files in the folders by semantic groups:
features
├──frontend
│ └──auth
│ └──login.feature
└──backend
└──auth
└──login.feature
And tests for this features could be organized in the next manner:
tests
└──conftest.py
└──functional
│ └──__init__.py
│ └──conftest.py
│ │ └── "User step library used by descendant tests"
│ │
│ │ from steps.auth.given import *
│ │ from steps.auth.when import *
│ │ from steps.auth.then import *
│ │
│ │ from steps.order.given import *
│ │ from steps.order.when import *
│ │ from steps.order.then import *
│ │
│ │ from steps.browser.given import *
│ │ from steps.browser.when import *
│ │ from steps.browser.then import *
│ │
│ └──frontend_auth.feature -> ../../features/frontend/auth.feature
│ └──backend_auth.feature -> ../../features/backend/auth.feature
...
Step definitions could be organized in the next way
steps
└──auth
│ └── given.py
│ │ └── """User auth step definitions"""
│ │ from pytest import fixture
│ │ from pytest_bdd import given, when, then, step
│ │
│ │ @fixture
│ │ def credentials():
│ │ return 'test_login', 'test_very_secure_pass'
│ │
│ │ @given('User login into application')
│ │ def user_login(credentials):
│ │ ...
│ └── when.py
│ └── then.py
└──order
│ └── given.py
│ └── when.py
│ └── then.py
└──browser
│ └── ...
...
To make links between feature files at features directory and test directory there are few options (for more information please investigate project tests):
Symlinks
.desktop files
.webloc files
.url files
Note
Link files also could be used to load features by http://
Features¶
Note
Features below are part of end-to-end test suite; You always could find most specific use cases of pytest-bdd-ng by investigation of its regression test suite https://github.com/elchupanebrej/pytest-bdd-ng/tree/default/tests
Feature: Gherkin steps are executed by pytest
Scenario: Steps are executed by corresponding step keyword decorator
Given File "steps.feature" with content:
"""gherkin
Feature: Steps are executed by corresponding step keyword decorator
Scenario:
Given Step counter
* Step is executed by plain step decorator
And Step is executed by plain step decorator
But Step is executed by plain step decorator
Given Step is executed by given step decorator
And Step is executed by given step decorator
But Step is executed by given step decorator
When Step is executed by when step decorator
And Step is executed by when step decorator
But Step is executed by when step decorator
Then Step is executed by then step decorator
And Step is executed by then step decorator
But Step is executed by then step decorator
Then there are passed steps by kind:
|step|given|when|then|
| 3| 3| 3| 3|
"""
And File "conftest.py" with content:
"""python
from operator import attrgetter
from pytest_bdd import given, when, then, step
from pytest_bdd.utils import compose
@given('Step counter', target_fixture='step_counter')
def step_counter():
yield {'step': 0, 'given': 0,'when': 0,'then': 0,}
@step('Step is executed by plain step decorator')
def plain_step(step_counter):
step_counter['step'] += 1
@given('Step is executed by given step decorator')
def given_step(step_counter):
step_counter['given'] += 1
@when('Step is executed by when step decorator')
def when_step(step_counter):
step_counter['when'] += 1
@then('Step is executed by then step decorator')
def then_step(step_counter):
step_counter['then'] += 1
@then('there are passed steps by kind:')
def check_step_counter(step, step_counter):
oracle_results_kwargs = map(attrgetter("value"), step.data_table.rows[0].cells)
oracle_results_kwargs_values = map(compose(int, attrgetter("value")), step.data_table.rows[1].cells)
oracle_result = dict(zip(oracle_results_kwargs, oracle_results_kwargs_values))
assert oracle_result == step_counter
"""
When run pytest
Then pytest outcome must contain tests with statuses:
|passed|
| 1|
Scenario: Steps could be executed by aliased step keyword decorator
Could be useful to declare the same fixtures or steps with
different names for better readability. In order to use the same step
function with multiple step names simply decorate it multiple times.
Given File "steps.feature" with content:
"""gherkin
Feature: Steps could be executed by aliased step keyword decorator
Scenario:
Given Step counter
* Step is executed by aliased step decorator
Given Step is executed by aliased step decorator
When Step is executed by aliased step decorator
Then Step is executed by aliased step decorator
Then there are "4" passed aliased steps
"""
And File "conftest.py" with content:
"""python
from pytest_bdd import given, when, then, step
@given('Step counter', target_fixture='step_counter')
def step_counter():
yield {'steps_count': 0}
@step('Step is executed by aliased step decorator')
@given('Step is executed by aliased step decorator')
@when('Step is executed by aliased step decorator')
@then('Step is executed by aliased step decorator')
def aliased_step(step_counter):
step_counter['steps_count'] += 1
@then(
'there are "{int}" passed aliased steps',
anonymous_group_names=('oracle_steps',),
)
def then_step(step_counter, oracle_steps):
assert step_counter['steps_count'] == oracle_steps
"""
When run pytest
Then pytest outcome must contain tests with statuses:
|passed|
| 1|
Rule: Steps could be executed by liberal step keyword decorator
Background:
Given File "steps.feature" with content:
"""gherkin
Feature: Steps could be executed by liberal step keyword decorator
Scenario:
Given Step counter
* Step is executed by liberal step decorator
Given Step is executed by liberal step decorator
When Step is executed by liberal step decorator
Then Step is executed by liberal step decorator
* Step is executed by liberal given decorator
Given Step is executed by liberal given decorator
When Step is executed by liberal given decorator
Then Step is executed by liberal given decorator
* Step is executed by liberal when decorator
Given Step is executed by liberal when decorator
When Step is executed by liberal when decorator
Then Step is executed by liberal when decorator
* Step is executed by liberal then decorator
Given Step is executed by liberal then decorator
When Step is executed by liberal then decorator
Then Step is executed by liberal then decorator
Then there are "16" passed liberal steps
"""
Scenario: Same step is used with different keywords
Given File "conftest.py" with content:
"""python
from pytest_bdd import given, when, then, step
@given('Step counter', target_fixture='step_counter')
def step_counter():
yield {'steps_count': 0}
@step('Step is executed by liberal step decorator', liberal=True)
@given('Step is executed by liberal given decorator', liberal=True)
@when('Step is executed by liberal when decorator', liberal=True)
@then('Step is executed by liberal then decorator', liberal=True)
def liberal_step(step_counter):
step_counter['steps_count'] += 1
@then(
'there are "{int}" passed liberal steps',
anonymous_group_names=('oracle_steps',),
)
def then_step(step_counter, oracle_steps):
assert step_counter['steps_count'] == oracle_steps
"""
When run pytest
Then pytest outcome must contain tests with statuses:
|passed|
| 1|
Scenario: Keyworded steps could be treated as liberal by pytest command line option
Given File "conftest.py" with content:
"""python
from pytest_bdd import given, when, then, step
@given('Step counter', target_fixture='step_counter')
def step_counter():
yield {'steps_count': 0}
@step('Step is executed by liberal step decorator')
@given('Step is executed by liberal given decorator')
@when('Step is executed by liberal when decorator')
@then('Step is executed by liberal then decorator')
def liberal_step(step_counter):
step_counter['steps_count'] += 1
@then(
'there are "{int}" passed liberal steps',
anonymous_group_names=('oracle_steps',),
)
def then_step(step_counter, oracle_steps):
assert step_counter['steps_count'] == oracle_steps
"""
When run pytest
|cli_args| --liberal-steps|
Then pytest outcome must contain tests with statuses:
|passed|failed|
| 1| 0|
Feature: Scenario Outline examples could be tagged
Rule:
Background:
Given File "steps.feature" with content:
"""gherkin
Feature: Steps are executed by corresponding step keyword decorator
Scenario Outline:
Given I produce <outcome> test
@passed
Examples:
|outcome|
|passed |
@failed
Examples:
|outcome|
|failed |
@both
Examples:
|outcome|
|passed |
|failed |
"""
Given File "pytest.ini" with content:
"""ini
[pytest]
markers =
passed
failed
both
"""
And File "conftest.py" with content:
"""python
from pytest_bdd.compatibility.pytest import fail
from pytest_bdd import given
@given('I produce passed test')
def passing_step():
...
@given('I produce failed test')
def failing_step():
fail('Enforce fail')
"""
Example:
When run pytest
|cli_args|-m|passed|
Then pytest outcome must contain tests with statuses:
|passed|failed|
| 1| 0|
Example:
When run pytest
|cli_args|-m|failed|
Then pytest outcome must contain tests with statuses:
|passed|failed|
| 0| 1|
Example:
When run pytest
|cli_args|-m|passed or failed|
Then pytest outcome must contain tests with statuses:
|passed|failed|
| 1| 1|
Example:
When run pytest
|cli_args|-m|not both|
Then pytest outcome must contain tests with statuses:
|passed|failed|
| 1| 1|
Example:
When run pytest
|cli_args|-m|both|
Then pytest outcome must contain tests with statuses:
|passed|failed|
| 1| 1|
Example:
When run pytest
Then pytest outcome must contain tests with statuses:
|passed|failed|
| 2| 2|
Rule: Mixing tags on feature & examples level
Background:
Given File "steps.feature" with content:
"""gherkin
@feature_tag
Feature: Steps are executed by corresponding step keyword decorator
Scenario Outline:
Given I produce <outcome> test
Examples:
|outcome|
|passed |
@examples_tag
Examples:
|outcome|
|failed |
"""
Given File "pytest.ini" with content:
"""ini
[pytest]
markers =
feature_tag
examples_tag
"""
And File "conftest.py" with content:
"""python
from pytest_bdd.compatibility.pytest import fail
from pytest_bdd import given
@given('I produce passed test')
def passing_step():
...
@given('I produce failed test')
def failing_step():
fail('Enforce fail')
"""
Example:
When run pytest
|cli_args|-m|feature_tag|
Then pytest outcome must contain tests with statuses:
|passed|failed|
| 1| 1|
Example:
When run pytest
|cli_args|-m|examples_tag|
Then pytest outcome must contain tests with statuses:
|passed|failed|
| 0| 1|
Example:
When run pytest
|cli_args|-m|not feature_tag|
Then pytest outcome must contain tests with statuses:
|passed|failed|
| 0| 0|
Example:
When run pytest
|cli_args|-m|not examples_tag|
Then pytest outcome must contain tests with statuses:
|passed|failed|
| 1| 0|
Example:
When run pytest
|cli_args|-m|feature_tag|--collect-only|
Then pytest outcome must match lines:
|collected 2 items|
Example:
When run pytest
|cli_args|-m|examples_tag|--collect-only|
Then pytest outcome must match lines:
|collected 2 items / 1 deselected / 1 selected|
Example:
When run pytest
|cli_args|-m|not feature_tag|--collect-only|
Then pytest outcome must match lines:
|collected 2 items / 2 deselected*|
Example:
When run pytest
|cli_args|-m|not examples_tag|--collect-only|
Then pytest outcome must match lines:
|collected 2 items / 1 deselected / 1 selected|
Scenario decorator¶
Feature files auto-collection could be disabled by use –disable-feature-autoload cli option or disable_feature_autoload pytest.ini option. In this case there mechanism to use features from test_*.py pytest files: functions decorated with the scenario decorator behave like a normal test function, and they will be executed after all scenario steps.
from pytest_bdd import scenario, given, when, then
@scenario('publish_article.feature', 'Publishing the article')
def test_publish(browser):
assert article.title in browser.html
Note
It is however encouraged to try as much as possible to have your logic only inside the Given, When, Then steps.
Step parameters¶
Often it’s possible to reuse steps giving them a parameter(s). This allows to have single implementation and multiple use, so less code. Also opens the possibility to use same step twice in single scenario and with different arguments! And even more, there are several types of step parameter parsers at your disposal (idea taken from behave implementation):
- heuristic (default)
Tries to select right parser between string, cucumber_expression, cfparse and re. Any object that supports __str__ interface and does not support parser interface will be wrapped with this parser
- parse (based on: pypi_parse)
Provides a simple parser that replaces regular expressions for step parameters with a readable syntax like
{param:Type}. The syntax is inspired by the Python builtinstring.format()function. Step parameters must use the named fields syntax of pypi_parse in step definitions. The named fields are extracted, optionally type converted and then used as step function arguments. Supports type conversions by using type converters passed via extra_types- cfparse (extends: pypi_parse, based on: pypi_parse_type)
Provides an extended parser with “Cardinality Field” (CF) support. Automatically creates missing type converters for related cardinality as long as a type converter for cardinality=1 is provided. Supports parse expressions like: *
{values:Type+}(cardinality=1..N, many) *{values:Type*}(cardinality=0..N, many0) *{value:Type?}(cardinality=0..1, optional) Supports type conversions (as above).- re
This uses full regular expressions to parse the clause text. You will need to use named groups “(?P<name>…)” to define the variables pulled from the text and passed to your
step()function. Type conversion can only be done via converters step decorator argument (see example below).- string
This can be considered as a null or exact parser. It parses no parameters and matches the step name by equality of strings.
- cucumber_expression (based on: pypi_cucumber_expressions)
Cucumber Expressions is an alternative to Regular Expressions with a more intuitive syntax.
- cucumber_regular_expression (based on: pypi_cucumber_expressions)
Alternative regular expression step parser
Parsers except string, as well as their optional arguments are specified like:
for cfparse parser:
from pytest_bdd import parsers
@given(
parsers.cfparse(
"there are {start:Number} cucumbers",
extra_types=dict(Number=int)
),
target_fixture="start_cucumbers",
)
def start_cucumbers(start):
return dict(start=start, eat=0)
or using cfparse Parser (or parse) directly:
from parse_type.cfparse import Parser as cfparse
@given(
cfparse(
"there are {start:Number} cucumbers",
extra_types=dict(Number=int)
),
target_fixture="start_cucumbers",
)
def start_cucumbers(start):
return dict(start=start, eat=0)
for re parser
from pytest_bdd import parsers
@given(
parsers.re(r"there are (?P<start>\d+) cucumbers"),
converters=dict(start=int),
target_fixture="start_cucumbers",
)
def start_cucumbers(start):
return dict(start=start, eat=0)
or using compiled regular expression directly:
import re
@given(
re.compile(r"there are (?P<start>\d+) cucumbers"),
converters=dict(start=int),
target_fixture="start_cucumbers",
)
def start_cucumbers(start):
return dict(start=start, eat=0)
for cucumber_expression:
from pytest_bdd import parsers
@given(
parsers.cucumber_expression("there are {int} cucumbers"),
anonymous_group_names=('start',),
target_fixture="start_cucumbers",
)
def start_cucumbers(start):
return dict(start=start, eat=0)
or using CucumberExpression directly:
from cucumber_expressions.expression import CucumberExpression
from cucumber_expressions.parameter_type_registry import ParameterTypeRegistry
@given(
CucumberExpression("there are {int} cucumbers", parameter_type_registry=ParameterTypeRegistry()),
anonymous_group_names=('start',),
target_fixture="start_cucumbers",
)
def start_cucumbers(start):
return dict(start=start, eat=0)
Note
anonymous_group_names step parameter is used to give names for non-named cucumber/regular expression groups.
Example:
Feature: Step arguments
Scenario: Arguments for given, when, then
Given there are 5 cucumbers
When I eat 3 cucumbers
And I eat 2 cucumbers
Then I should have 0 cucumbers
The code will look like:
import re
from parse import Parser as parse
from pytest_bdd import scenario, given, when, then
@scenario("arguments.feature", "Arguments for given, when, then")
def test_arguments():
pass
@given(parse("there are {start:d} cucumbers"), target_fixture="start_cucumbers")
def start_cucumbers(start):
return dict(start=start, eat=0)
@when(parse("I eat {eat:d} cucumbers"))
def eat_cucumbers(start_cucumbers, eat):
start_cucumbers["eat"] += eat
@then(parse("I should have {left:d} cucumbers"))
def should_have_left_cucumbers(start_cucumbers, start, left):
assert start_cucumbers['start'] == start
assert start - start_cucumbers['eat'] == left
Example code also shows possibility to pass argument converters which may be useful if you need to postprocess step arguments after the parser.
You can implement your own step parser. It’s interface is quite simple. The code can looks like:
import re
from pytest_bdd import given, parsers
class MyParser(parsers.StepParser):
"""Custom parser."""
def __init__(self, name, **kwargs):
"""Compile regex."""
super().__init__(name)
self.regex = re.compile(re.sub("%(.+)%", "(?P<\1>.+)", self.name), **kwargs)
def parse_arguments(self, name):
"""Get step arguments.
:return: `dict` of step arguments
"""
return self.regex.match(name).groupdict()
def is_matching(self, name):
"""Match given name with the step name."""
return bool(self.regex.match(name))
@given(parsers.parse("there are %start% cucumbers"), target_fixture="start_cucumbers")
def start_cucumbers(start):
return dict(start=start, eat=0)
Step arguments could be defined without parsing¶
If you want specify some default values for parameters without parsing them (useful for step aliases), you could do:
@given("I have default defined param", param_defaults={'default_param': 'foo'}, target_fixture='foo_fixture')
def save_fixture(default_param):
return default_param
Step arguments are injected into step context¶
Step arguments are injected into step context and could be used as normal fixtures with the names equal to the names of the arguments by default. This opens a number of possibilities:
you can access step’s argument as a fixture in other step function just by mentioning it as an argument (just like any other pytest fixture)
if the name of the step argument clashes with existing fixture, it will be overridden by step’s argument value; this way you can set/override the value for some fixture deeply inside of the fixture tree in a ad-hoc way by just choosing the proper name for the step argument.
This behavior is same to:
@given(
'I have a "{foo}", "{bar}", "{fizz}", "{buzz}" parameters few of which are accepted by wild pattern',
params_fixtures_mapping={...: ...}
)
def step(foo, bar, fizz, buzz):
...
But this behavior could be changed; For example you want to rename some parameters and left other as-is. Ellipsis instance means all present attributes, but not listed directly.
@given(
'I have a "{foo}", "{bar}", "{fizz}", "{buzz}" parameters few of which are accepted by wild pattern',
params_fixtures_mapping={'foo': 'cool_foo', 'bar': 'nice_bar', ...: ...}
)
def step(cool_foo, nice_bar, fizz, buzz):
...
Or don’t inject parameters as fixtures at all:
@given('I have a "{foo}", "{bar}", "{fizz}", "{buzz}" parameters few of which are accepted by wild pattern',
params_fixtures_mapping={...: None})
def step(foo, bar, fizz, buzz):
...
Parameters still could be used in steps, but they are not injected! If you would like to inject just some subset of parameters - set of parameters could be used:
@given('I have a "{foo}", "{bar}", "{fizz}", "{buzz}" parameters few of which are accepted by wild pattern',
params_fixtures_mapping={'fizz', 'buzz'})
def step(foo, bar, fizz, buzz):
...
Override fixtures by outgoing step results¶
Dependency injection is not a panacea if you have complex structure of your test setup data. Sometimes there’s a need such a given step which would imperatively change the fixture only for certain test (scenario), while for other tests it will stay untouched. To allow this, special parameter target_fixture exists in the decorator:
from pytest_bdd import given
@pytest.fixture
def foo():
return "foo"
@given("I have injecting given", target_fixture="foo")
def injecting_given():
return "injected foo"
@then('foo should be "injected foo"')
def foo_is_foo(foo):
assert foo == 'injected foo'
Feature: Target fixture
Scenario: Test given fixture injection
Given I have injecting given
Then foo should be "injected foo"
In this example existing fixture foo will be overridden by given step I have injecting given only for scenario it’s used in.
Sometimes it is also useful to let when and then steps to provide a fixture as well. A common use case is when we have to assert the outcome of an HTTP request:
# test_blog.py
from pytest_bdd import scenarios, given, when, then
from my_app.models import Article
test_cukes = scenarios("blog.feature")
@given("there is an article", target_fixture="article")
def there_is_an_article():
return Article()
@when("I request the deletion of the article", target_fixture="request_result")
def there_should_be_a_new_article(article, http_client):
return http_client.delete(f"/articles/{article.uid}")
@then("the request should be successful")
def article_is_published(request_result):
assert request_result.status_code == 200
# blog.feature
Feature: Blog
Scenario: Deleting the article
Given there is an article
When I request the deletion of the article
Then the request should be successful
Also it’s possible to override multiple fixtures in one step using target_fixtures parameter:
@given("some compound fixture", target_fixtures=["fixture_a","fixture_b"])
def __():
return "fixture_a_value", "fixture_b_value"
Loading whole feature files¶
If you have relatively large set of feature files, it’s boring to manually bind scenarios to the tests using the scenario decorator(in case if you don’t want use feature auto-load). Of course with the manual approach you get all the power to be able to additionally parametrize the test, give the test function a nice name, document it, etc, but in the majority of the cases you don’t need that. Instead you want to bind all scenarios found in the feature folder(s) recursively automatically.
Scenarios shortcut¶
First option is scenarios helper.
from pytest_bdd import scenarios
# assume 'features' subfolder is in this file's directory
test_cukes = scenarios('features')
That’s all you need to do to bind all scenarios found in the features folder! Note that you can pass multiple paths, and those paths can be either feature files or feature folders.
from pytest_bdd import scenarios
# pass multiple paths/files
test_cukes = scenarios('features', 'other_features/some.feature', 'some_other_features')
But what if you need to manually bind certain scenario, leaving others to be automatically bound? Just write your scenario in a normal way, but filter out scenario, and bind it manually:
from pytest_bdd import scenario, scenarios
# assume 'features' subfolder is in this file's directory
test_cukes = scenarios('features', filter_ = lambda: config, feature, scenario: scenario.name != 'Test something')
@scenario('features/some.feature', 'Test something')
def test_something():
pass
In the example above test_something scenario binding will be kept manual, other scenarios found in the features folder will be bound automatically.
Both scenario or scenarios could be used as decorators or as operator calls. Also they could be inlined:
from pytest_bdd import scenario, scenarios
test_features = scenarios('features', return_test_decorator=False)
test_specific_scenario = scenario('features/some.feature', 'Test something', return_test_decorator=False)
Both scenario and scenarios functions could use http/https URIs to get features from remote servers (and be integrated with tools like Hiptest)
Test setup¶
Test setup is implemented within the Given section. Even though these steps are executed imperatively to apply possible side-effects, pytest-bdd-ng is trying to benefit of the PyTest fixtures which is based on the dependency injection and makes the setup more declarative style.
@given("I have a beautiful article", target_fixture="article")
def article():
return Article(is_beautiful=True)
The target PyTest fixture “article” gets the return value and any other step can depend on it.
Feature: The power of PyTest
Scenario: Symbolic name across steps
Given I have a beautiful article
When I publish this article
When step is referring the article to publish it.
@when("I publish this article")
def publish_article(article):
article.publish()
Many other BDD toolkits operate a global context and put the side effects there. This makes it very difficult to implement the steps, because the dependencies appear only as the side-effects in the run-time and not declared in the code. The publish article step has to trust that the article is already in the context, has to know the name of the attribute it is stored there, the type etc.
In pytest-bdd-ng you just declare an argument of the step function that it depends on and the PyTest will make sure to provide it.
Still side effects can be applied in the imperative style by design of the BDD.
Feature: News website
Scenario: Publishing an article
Given I have a beautiful article
And my article is published
Functional tests can reuse your fixture libraries created for the unit-tests and upgrade them by applying the side effects.
@pytest.fixture
def article():
return Article(is_beautiful=True)
@given("I have a beautiful article")
def i_have_a_beautiful_article(article):
pass
@given("my article is published")
def published_article(article):
article.publish()
return article
This way side-effects were applied to our article and PyTest makes sure that all steps that require the “article” fixture will receive the same object. The value of the “published_article” and the “article” fixtures is the same object.
Fixtures are evaluated only once within the PyTest scope and their values are cached.
Reusing steps¶
It is possible to define some common steps in the parent conftest.py and simply expect them in the child test file.
common_steps.feature:
Scenario: All steps are declared in the conftest
Given I have a bar
Then bar should have value "bar"
conftest.py:
from pytest_bdd import given, then
@given("I have a bar", target_fixture="bar")
def bar():
return "bar"
@then('bar should have value "bar"')
def bar_is_bar(bar):
assert bar == "bar"
test_common.py:
@scenario("common_steps.feature", "All steps are declared in the conftest")
def test_conftest():
pass
There are no definitions of the steps in the test file. They were collected from the parent conftest.py.
Default steps¶
Here is the list of steps that are implemented inside of the pytest-bdd:
- given
trace - enters the pdb debugger via pytest.set_trace()
- when
trace - enters the pdb debugger via pytest.set_trace()
- then
trace - enters the pdb debugger via pytest.set_trace()
Feature file paths¶
By default, pytest-bdd-ng will use current module’s path as base path for finding feature files, but this behaviour can be changed in the pytest configuration file (i.e. pytest.ini, tox.ini or setup.cfg) by declaring the new base path in the bdd_features_base_dir key. The path is interpreted as relative to the pytest root directory. You can also override features base path on a per-scenario basis, in order to override the path for specific tests.
pytest.ini:
[pytest]
bdd_features_base_dir = features/
tests/test_publish_article.py:
from pytest_bdd import scenario
@scenario("foo.feature", "Foo feature in features/foo.feature")
def test_foo():
pass
@scenario(
"foo.feature",
"Foo feature in tests/local-features/foo.feature",
features_base_dir="./local-features/",
)
def test_foo_local():
pass
The features_base_dir parameter can also be passed to the @scenario decorator.
Localization¶
pytest-bdd-ng supports all localizations which Gherkin does
Hooks¶
pytest-bdd-ng exposes several pytest hooks which might be helpful building useful reporting, visualization, etc on top of it:
pytest_bdd_before_scenario(request, feature, scenario) - Called before scenario is executed
pytest_bdd_run_scenario(request, feature, scenario) - Execution scenario protocol
pytest_bdd_after_scenario(request, feature, scenario) - Called after scenario is executed (even if one of steps has failed)
pytest_bdd_before_step(request, feature, scenario, step, step_func) - Called before step function is executed and it’s arguments evaluated
pytest_bdd_run_step(request, feature, scenario, step, previous_step) - Execution step protocol
pytest_bdd_before_step_call(request, feature, scenario, step, step_func, step_func_args) - Called before step function is executed with evaluated arguments
pytest_bdd_after_step(request, feature, scenario, step, step_func, step_func_args) - Called after step function is successfully executed
pytest_bdd_step_error(request, feature, scenario, step, step_func, step_func_args, exception) - Called when step function failed to execute
pytest_bdd_step_func_lookup_error(request, feature, scenario, step, exception) - Called when step lookup failed
pytest_bdd_match_step_definition_to_step(request, feature, scenario, step, previous_step) - Called to match step to step definition
pytest_bdd_get_step_caller(request, feature, scenario, step, step_func, step_func_args, step_definition) - Called to get step caller. For example could be used to make steps async
pytest_bdd_get_step_dispatcher(request, feature, scenario) - Provide alternative approach to execute scenario steps
Fixtures¶
pytest-bdd-ng exposes several plugin fixtures to give more testing flexibility
bdd_example - The current scenario outline parametrization.
attach - Fixture to allow attach files to Gherkin report
parameter_type_registry - Contains registry of user-defined types used in Cucumber expressions
step_registry - Contains registry of all user-defined steps
step_matcher- Contains matcher to help find step definition for selected step of scenario
steps_left - Current scenario steps left to execute; Allow inject steps to execute:
from collections import deque
from pytest_bdd.model import UserStep
from pytest_bdd import when
@when('I inject step "{keyword}" "{step_text}')
def inject_step(steps_left: deque, keyword, step_text, scenario):
steps_left.appendleft(UserStep(text=step_text, keyword=keyword, scenario=scenario))
StructBDD¶
Gherkin itself isn’t a perfect tool to describe complex Data Driven Scenarios with alternative paths to execute test. For example it doesn’t support next things:
Few backgrounds per scenario
Alternative flows for scenario to setup same state
Alternative flows to describe same behavior defined by different steps
Usage of parameters inside Backgrounds
Joining of parameter tables, so full Cartesian product of parameters has to be listed in Examples
Example tables on different scenario levels
For such scenarios StructBDD DSL was developed. It independent on underlying data format, but supports most common formats for DSL development: YAML, Hocon, TOML, JSON5, HJSON out the box.
Steps could be defined as usual, and scenarios have different options. Let see.
steps.bdd.yaml
Name: Steps are executed one by one
Description: |
Steps are executed one by one. Given and When sections
are not mandatory in some cases.
Steps:
- Step:
Name: Executed step by step
Description: Scenario description
Steps:
- I have a foo fixture with value "foo"
- And: there is a list
- When: I append 1 to the list
- Step:
Action: I append 2 to the list
Type: And
- Alternative:
- Step:
Steps:
- And: I append 3 to the list
- Then: foo should have value "foo"
- But: the list should be [1, 2, 3]
- Step:
Steps:
- And: I append 4 to the list
- Then: foo should have value "foo"
- But: the list should be [1, 2, 4]
Alternative steps produce separate test launches for every of flows. If alternative steps are defined on different levels - there would be Cartesian product of tests for every alternative step.
Scenario could be imported as usual, but with specified parser:
from textwrap import dedent
from pytest_bdd import given, when, then, scenario
from pytest_bdd.parser import StructBDDParser
from functools import partial
kind = StructBDDParser.KIND.YAML
@scenario(f"steps.bdd.{kind}", "Executed step by step", parser=partial(StructBDDParser, kind=kind)
def test_steps(feature):
pass
Another option is to inject built scenario directly:
from pytest_bdd.struct_bdd.model import Step, Table
test_cukes = Step(
name="Examples are substituted",
steps=[
Step(type='Given', action='I have <have> cucumbers'),
Step(type='And', action='I eat <eat> cucumbers'),
Step(type='Then', action='I have <left> cucumbers')
],
examples=[
Table(
parameters=['have', 'eat', 'left'],
values=[
['12', 5, 7.0],
["8.0", 3.0, "5"]
]
)
]
)
There is also an option to build Step from dict(and use your own file format/preprocessor)
from pytest_bdd.struct_bdd.model import Step
cukes = Step.parse_obj(
dict(
Name="Examples are substituted",
Steps=[
dict(Given='I have <have> cucumbers'),
dict(And='I eat <eat> cucumbers'),
dict(Then='I have <left> cucumbers')
],
Examples=[
dict(
Table=dict(
Parameters=['have', 'eat', 'left'],
Values=[
['12', 5, 7.0],
["8.0", 3.0, "5"]
]
)
)
]
)
)
@cukes
def test(feature:Feature, scenario):
assert feature.name == "Examples are substituted"
Example tables could be joined:
Tags:
- TopTag
Name: StepName
Action: "Do first <HeaderA>, <HeaderB>, <HeaderC>"
Examples:
- Join:
- Table:
Tags:
- ExampleTagA
Parameters:
[ HeaderA, HeaderB ]
Values:
- [ A1, B1]
- [ A2, B2]
- Table:
Tags:
- ExampleTagB
Parameters:
[ HeaderB, HeaderC ]
Values:
- [ B1, C1 ]
- [ B2, C2 ]
- [ B3, C3 ]
Steps: []
Install StructBDD:
pip install pytest-bdd-ng[struct_bdd]
Reporting¶
It’s important to have nice reporting out of your bdd tests. Cucumber introduced some kind of standard for json format which can be used for, for example, by this Jenkins plugin.
To have an output in json format:
pytest --cucumberjson=<path to json report>
This will output an expanded (meaning scenario outlines will be expanded to several scenarios) cucumber format.
To enable gherkin-formatted output on terminal, use
pytest --gherkin-terminal-reporter
Allure reporting is also in place https://docs.qameta.io/allure and based on allure-pytest https://pypi.org/project/allure-pytest/ plugin. Usage is same.
To install plugin¶
pip install pytest-bdd-ng[allure]
Test code generation helpers¶
For newcomers it’s sometimes hard to write all needed test code without being frustrated. To simplify their life, simple code generator was implemented. It allows to create fully functional but of course empty tests and step definitions for given a feature file. It’s done as a separate console script provided by pytest-bdd package:
pytest --generate --feature <feature file name> .. <feature file nameN>
It will print the generated code to the standard output so you can easily redirect it to the file:
pytest --generate --feature features/some.feature > tests/functional/test_some.py
Advanced code generation¶
For more experienced users, there’s smart code generation/suggestion feature. It will only generate the test code which is not yet there, checking existing tests and step definitions the same way it’s done during the test execution. The code suggestion tool is called via passing additional pytest arguments:
pytest --generate-missing --feature features tests/functional
The output will be like:
============================= test session starts ==============================
platform linux2 -- Python 2.7.6 -- py-1.4.24 -- pytest-2.6.2
plugins: xdist, pep8, cov, cache, bdd, bdd, bdd
collected 2 items
Scenario is not bound to any test: "Code is generated for scenarios which are not bound to any tests" in feature "Missing code generation" in /tmp/pytest-552/testdir/test_generate_missing0/tests/generation.feature
--------------------------------------------------------------------------------
Step is not defined: "I have a custom bar" in scenario: "Code is generated for scenario steps which are not yet defined(implemented)" in feature "Missing code generation" in /tmp/pytest-552/testdir/test_generate_missing0/tests/generation.feature
--------------------------------------------------------------------------------
Please place the code above to the test file(s):
@scenario('tests/generation.feature', 'Code is generated for scenarios which are not bound to any tests')
def test_Code_is_generated_for_scenarios_which_are_not_bound_to_any_tests():
"""Code is generated for scenarios which are not bound to any tests."""
@given("I have a custom bar")
def I_have_a_custom_bar():
"""I have a custom bar."""
As as side effect, the tool will validate the files for format errors, also some of the logic bugs, for example the ordering of the types of the steps.
Changelog¶
In-progress¶
Move documentation to Gherkin itself
Planned¶
Review report generation to be conform with official tools
Nested Rules support
Review after fix https://github.com/cucumber/gherkin/issues/126
Implement support of *.md files
Waiting for upstream issue https://github.com/cucumber/gherkin/pull/64
Support of messages:
Pending:
parse_error
undefined_parameter_type
Add support of python 3.12 at CI
Add mode to execute scenarios with missing/failing steps
Remove
Hide traceback for pytest code “__tracebackhide__ = True”
Generate documentation via https://github.com/jolly-good-toolbelt/sphinx_gherkindoc instead of direct use
Move sphinx-gherkindoc to official parser
Rework generation code to include new features directly
Generate code into dir structure aligned with proposed project layout
Test messages against
pytest-xdist at workers on different machines (sending back ndjson info https://codespeak.net/execnet/example/test_info.html#sending-channels-over-channels)
Investigate https://smarie.github.io/python-pytest-harvest/
pytest-rerunfailures
Parametrize step execution by different step realizations using https://smarie.github.io/python-pytest-cases/
Switch testdir to pytester after pytest<6.2 get EOL (python 3.8 and 3.9 get EOL)
Use Poetry
Contribute to messages repository with python model
Add support of native legacy cucumber-json
Test against https://github.com/cucumber/json-formatter
2.1.0¶
Using official cucmber ci-environment lib
Add validation for legacy cucumber.json output
Migrated to Pydantic 2
Add tests for PyPy
Using tox4
Added support of conditional hooks https://cucumber.io/docs/cucumber/api/?lang=java#conditional-hooks
Support of messages:
Done:
meta
gherkin_document
pickle
source
step_definition
test_case
test_case_finished
test_case_started
test_run_finished
test_run_started
test_step_finished
test_step_started
parameter_type
attachment
hook
2.0.0¶
Reviewed StructBdd step collection; no more
as_test/as_test_decoratorStep methods are needed directly usedDrop python 3.7
Move StructBDD model to Pydantic
Remove ast module usage by StructBDD
Test filters in scenario/scenarios to filter out not needed scenarios
.url,.desktopand.weblocfiles are collected from test directories, so scenario/scenarios usages is not necessaryLoad features/scenarios by url
Features are autoloaded by default; Feature autoload could be disabled by
--disable-feature-autoloadcli optionRelative feature paths are counted from pytest rootpath
No more injection of tests into module space; Tests have to be registered directly
Separate generation scripts were moved to pytest environment
scenariono more override collected scenarios; They have to be registered independently. Scenarios could be filtered out if needed.Added support of messages
Added support of cucumber expressions https://github.com/cucumber/cucumber-expressions#readme
It possible to name anonymous groups during step parsing
Remove legacy feature parser (and surplus features of it)
Remove outdated migration script
1.2.3¶
Features could be autoloaded by –feature-autoload cli option
Remove possibility to manually register imported steps; They are registered automatically
1.2.2¶
Add possibility to register imported steps
1.2.0¶
Make liberal step definitions conform with
1.1.2¶
Fixups
1.1.1¶
Added hook to alter scenario steps execution protocol
1.1.0¶
Added allure plugin extension for allure-pytest
Added StructBDD DSL
1.0.0¶
Default step parameter parser is switched to cfparse. String step name is compiled to cfparse
Step functions could get compiled instances of parse, cfparse and re.compile directly
Drop pytest 4
Drop python 3.6
Added tags support for Examples sections for original parser
Added joining by parameters between examples sections on different levels (and with fixtures) for original feature parser
Step could override multiple fixtures using
target_fixturesparameterDefault step parameters injection as fixtures behavior could be changed by
params_fixtures_mappingstep parameterStep definitions can have “yield” statements again (4.0 release broke it). They will be executed as normal fixtures: code after the yield is executed during teardown of the test. (youtux)
Show pass/fail status per step in Gherkin terminal reporter
Step definitions could be used independently from keyword by
stepdecoratorpytest_bdd_apply_tagwas removed;pytest_bdd_convert_tag_to_markswas added insteadFeature parser switched to official one
Changes
scenarioandscenariosfunction/decorator feature registration order. Both could be used as decoratorsMove scenario execution & step matching to hooks
Added possibility to operate steps stack via fixture
Other
Pre pytest-bdd-ng era¶
5.0.0¶
This release introduces breaking changes, please refer to the Migration from 4.x.x.
Rewrite the logic to parse Examples for Scenario Outlines. Now the substitution of the examples is done during the parsing of Gherkin feature files. You won’t need to define the steps twice like
@given("there are <start> cucumbers")and@given(parsers.parse("there are {start} cucumbers")). The latter will be enough.Removed
example_convertersfromscenario(...)signature. You should now use just theconvertersparameter forgiven,when,then.Removed
--cucumberjson-expandedand--cucumber-json-expandedoptions. Now the JSON report is always expanded.Removed
--gherkin-terminal-reporter-expandedoption. Now the terminal report is always expanded.
4.1.0¶
when and then steps now can provide a target_fixture, just like given does. Discussion at https://github.com/pytest-dev/pytest-bdd/issues/402.
Drop compatibility for python 2 and officially support only python >= 3.6.
Fix error when using –cucumber-json-expanded in combination with example_converters (marcbrossaissogeti).
Fix –generate-missing not correctly recognizing steps with parsers
4.0.2¶
Fix a bug that prevents using comments in the
Examples:section. (youtux)
4.0.1¶
Fixed performance regression introduced in 4.0.0 where collection time of tests would take way longer than before. (youtux)
4.0.0¶
This release introduces breaking changes, please refer to the Migration from 3.x.x.
Strict Gherkin option is removed (
@scenario()does not accept thestrict_gherkinparameter). (olegpidsadnyi)@scenario()does not accept the undocumented parametercaller_moduleanymore. (youtux)Given step is no longer a fixture. The scope parameter is also removed. (olegpidsadnyi)
Fixture parameter is removed from the given step declaration. (olegpidsadnyi)
pytest_bdd_step_validation_errorhook is removed. (olegpidsadnyi)Fix an error with pytest-pylint plugin #374. (toracle)
Fix pytest-xdist 2.0 compatibility #369. (olegpidsadnyi)
Fix compatibility with pytest 6
--import-mode=importliboption. (youtux)
3.4.0¶
Parse multiline steps according to the gherkin specification #365.
3.3.0¶
Drop support for pytest < 4.3.
Fix a Python 4.0 bug.
Fix
pytest --generate-missingfunctionality being broken.Fix problematic missing step definition from strings containing quotes.
Implement parsing escaped pipe characters in outline parameters (Mark90) #337.
Disable the strict Gherkin validation in the steps generation (v-buriak) #356.
3.2.1¶
Fix regression introduced in 3.2.0 where pytest-bdd would break in presence of test items that are not functions.
3.2.0¶
Fix Python 3.8 support
Remove code that rewrites code. This should help with the maintenance of this project and make debugging easier.
3.1.1¶
Allow unicode string in
@given()step names when using python2. This makes the transition of projects from python 2 to 3 easier.
3.1.0¶
Drop support for pytest < 3.3.2.
Step definitions generated by
$ pytest-bdd generatewill now raiseNotImplementedErrorby default.@given(...)no longer accepts regex objects. It was deprecated long ago.Improve project testing by treating warnings as exceptions.
pytest_bdd_step_validation_errorwill now always receivestep_func_argsas defined in the signature.
3.0.2¶
Add compatibility with pytest 4.2 (sliwinski-milosz) #288.
3.0.1¶
Minimal supported version of pytest is now 2.9.0 as lower versions do not support bool type ini options (sliwinski-milosz) #260
Fix RemovedInPytest4Warning warnings (sliwinski-milosz) #261.
3.0.0¶
Fixtures pytestbdd_feature_base_dir and pytestbdd_strict_gherkin have been removed. Check the Migration of your tests from versions 2.x.x for more information (sliwinski-milosz) #255
Fix step definitions not being found when using parsers or converters after a change in pytest (youtux) #257
2.21.0¶
Gherkin terminal reporter expanded format (pauk-slon)
2.20.0¶
Added support for But steps (olegpidsadnyi)
Fixed compatibility with pytest 3.3.2 (olegpidsadnyi)
MInimal required version of pytest is now 2.8.1 since it doesn’t support earlier versions (olegpidsadnyi)
2.19.0¶
Added –cucumber-json-expanded option for explicit selection of expanded format (mjholtkamp)
Step names are filled in when –cucumber-json-expanded is used (mjholtkamp)
2.18.2¶
Fix check for out section steps definitions for no strict gherkin feature
2.18.1¶
Relay fixture results to recursive call of ‘get_features’ (coddingtonbear)
2.18.0¶
Add gherkin terminal reporter (spinus + thedrow)
2.17.2¶
Fix scenario lines containing an
@being parsed as a tag. (The-Compiler)
2.17.1¶
Add support for pytest 3.0
2.17.0¶
Fix FixtureDef signature for newer pytest versions (The-Compiler)
Better error explanation for the steps defined outside of scenarios (olegpidsadnyi)
Add a
pytest_bdd_apply_taghook to customize handling of tags (The-Compiler)Allow spaces in tag names. This can be useful when using the
pytest_bdd_apply_taghook with tags like@xfail: Some reason.
2.16.1¶
Cleaned up hooks of the plugin (olegpidsadnyi)
Fixed report serialization (olegpidsadnyi)
2.16.0¶
Fixed deprecation warnings with pytest 2.8 (The-Compiler)
Fixed deprecation warnings with Python 3.5 (The-Compiler)
2.15.0¶
Add examples data in the scenario report (bubenkoff)
2.14.5¶
Properly parse feature description (bubenkoff)
2.14.3¶
Avoid potentially random collection order for xdist compartibility (bubenkoff)
2.14.1¶
Pass additional arguments to parsers (bubenkoff)
2.14.0¶
Add validation check which prevents having multiple features in a single feature file (bubenkoff)
2.13.1¶
Allow mixing feature example table with scenario example table (bubenkoff, olegpidsadnyi)
2.13.0¶
Feature example table (bubenkoff, sureshvv)
2.12.2¶
Make it possible to relax strict Gherkin scenario validation (bubenkoff)
2.11.3¶
Fix minimal six version (bubenkoff, dustinfarris)
2.11.1¶
Mention step type on step definition not found errors and in code generation (bubenkoff, lrowe)
2.11.0¶
Prefix step definition fixture names to avoid name collisions (bubenkoff, lrowe)
2.10.0¶
Make feature and scenario tags to be fully compartible with pytest markers (bubenkoff, kevinastone)
2.9.1¶
Fixed FeatureError string representation to correctly support python3 (bubenkoff, lrowe)
2.9.0¶
Added possibility to inject fixtures from given keywords (bubenkoff)
2.8.0¶
Added hook before the step is executed with evaluated parameters (olegpidsadnyi)
2.7.2¶
Correct base feature path lookup for python3 (bubenkoff)
2.7.1¶
Allow to pass
scopeforgivensteps (bubenkoff, sureshvv)
2.7.0¶
Implemented scenarios shortcut to automatically bind scenarios to tests (bubenkoff)
2.6.2¶
Parse comments only in the beginning of words (santagada)
2.6.1¶
Correctly handle pytest-bdd command called without the subcommand under python3 (bubenkoff, spinus)
Pluggable parsers for step definitions (bubenkoff, spinus)
2.5.3¶
Add after scenario hook, document both before and after scenario hooks (bubenkoff)
2.5.2¶
Fix code generation steps ordering (bubenkoff)
2.5.1¶
Fix error report serialization (olegpidsadnyi)
2.5.0¶
Fix multiline steps in the Background section (bubenkoff, arpe)
Code cleanup (olegpidsadnyi)
2.4.5¶
Fix unicode issue with scenario name (bubenkoff, aohontsev)
2.4.3¶
Fix unicode regex argumented steps issue (bubenkoff, aohontsev)
Fix steps timings in the json reporting (bubenkoff)
2.4.2¶
Recursion is fixed for the –generate-missing and the –feature parameters (bubenkoff)
2.4.1¶
Better reporting of a not found scenario (bubenkoff)
Simple test code generation implemented (bubenkoff)
Correct timing values for cucumber json reporting (bubenkoff)
Validation/generation helpers (bubenkoff)
2.4.0¶
Background support added (bubenkoff)
Fixed double collection of the conftest files if scenario decorator is used (ropez, bubenkoff)
2.3.3¶
Added timings to the cucumber json report (bubenkoff)
2.3.2¶
Fixed incorrect error message using e.argname instead of step.name (hvdklauw)
2.3.1¶
Implemented cucumber tags support (bubenkoff)
Implemented cucumber json formatter (bubenkoff, albertjan)
Added ‘trace’ keyword (bubenkoff)
2.1.2¶
Latest pytest compartibility fixes (bubenkoff)
2.1.1¶
Bugfixes (bubenkoff)
2.1.0¶
Implemented multiline steps (bubenkoff)
2.0.1¶
Allow more than one parameter per step (bubenkoff)
Allow empty example values (bubenkoff)
2.0.0¶
Pure pytest parametrization for scenario outlines (bubenkoff)
Argumented steps now support converters (transformations) (bubenkoff)
scenario supports only decorator form (bubenkoff)
Code generation refactoring and cleanup (bubenkoff)
1.0.0¶
Implemented scenario outlines (bubenkoff)
0.6.11¶
Fixed step arguments conflict with the fixtures having the same name (olegpidsadnyi)
0.6.9¶
Implemented support of Gherkin “Feature:” (olegpidsadnyi)
0.6.8¶
Implemented several hooks to allow reporting/error handling (bubenkoff)
0.6.6¶
Fixes to unnecessary mentioning of pytest-bdd package files in py.test log with -v (bubenkoff)
0.6.5¶
Compartibility with recent pytest (bubenkoff)
0.6.4¶
More unicode fixes (amakhnach)
0.6.3¶
Added unicode support for feature files. Removed buggy module replacement for scenario. (amakhnach)
0.6.2¶
Removed unnecessary mention of pytest-bdd package files in py.test log with -v (bubenkoff)
0.6.1¶
Step arguments in whens when there are no given arguments used. (amakhnach, bubenkoff)
0.6.0¶
Added step arguments support. (curzona, olegpidsadnyi, bubenkoff)
Added checking of the step type order. (markon, olegpidsadnyi)
0.5.2¶
Added extra info into output when FeatureError exception raises. (amakhnach)
0.5.0¶
Added parametrization to scenarios
Coveralls.io integration
Test coverage improvement/fixes
Correct wrapping of step functions to preserve function docstring
0.4.7¶
Fixed Python 3.3 support
0.4.6¶
Fixed a bug when py.test –fixtures showed incorrect filenames for the steps.
0.4.5¶
Fixed a bug with the reuse of the fixture by given steps being evaluated multiple times.
0.4.3¶
Update the license file and PYPI related documentation.
MIT License¶
Copyright (C) 2013-2023 Oleg Pidsadnyi, Anatoly Bubenkov and others
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.