.. _monkeypatching:

How to monkeypatch/mock modules and environments
================================================================

.. currentmodule:: pytest

Sometimes tests need to invoke functionality which depends
on global settings or which invokes code which cannot be easily
tested such as network access.  The ``monkeypatch`` fixture
helps you to safely set/delete an attribute, dictionary item or
environment variable, or to modify ``sys.path`` for importing.

The ``monkeypatch`` fixture provides these helper methods for safely patching and mocking
functionality in tests:

* :meth:`monkeypatch.setattr(obj, name, value, raising=True) <pytest.MonkeyPatch.setattr>`
* :meth:`monkeypatch.delattr(obj, name, raising=True) <pytest.MonkeyPatch.delattr>`
* :meth:`monkeypatch.setitem(mapping, name, value) <pytest.MonkeyPatch.setitem>`
* :meth:`monkeypatch.delitem(obj, name, raising=True) <pytest.MonkeyPatch.delitem>`
* :meth:`monkeypatch.setenv(name, value, prepend=None) <pytest.MonkeyPatch.setenv>`
* :meth:`monkeypatch.delenv(name, raising=True) <pytest.MonkeyPatch.delenv>`
* :meth:`monkeypatch.syspath_prepend(path) <pytest.MonkeyPatch.syspath_prepend>`
* :meth:`monkeypatch.chdir(path) <pytest.MonkeyPatch.chdir>`
* :meth:`monkeypatch.context() <pytest.MonkeyPatch.context>`


All modifications will be undone after the requesting
test function or fixture has finished. The ``raising``
parameter determines if a ``KeyError`` or ``AttributeError``
will be raised if the target of the set/deletion operation does not exist.

Consider the following scenarios:

1. Modifying the behavior of a function or the property of a class for a test e.g.
there is an API call or database connection you will not make for a test but you know
what the expected output should be. Use :py:meth:`monkeypatch.setattr <MonkeyPatch.setattr>` to patch the
function or property with your desired testing behavior. This can include your own functions.
Use :py:meth:`monkeypatch.delattr <MonkeyPatch.delattr>` to remove the function or property for the test.

2. Modifying the values of dictionaries e.g. you have a global configuration that
you want to modify for certain test cases. Use :py:meth:`monkeypatch.setitem <MonkeyPatch.setitem>` to patch the
dictionary for the test. :py:meth:`monkeypatch.delitem <MonkeyPatch.delitem>` can be used to remove items.

3. Modifying environment variables for a test e.g. to test program behavior if an
environment variable is missing, or to set multiple values to a known variable.
:py:meth:`monkeypatch.setenv <MonkeyPatch.setenv>` and :py:meth:`monkeypatch.delenv <MonkeyPatch.delenv>` can be used for
these patches.

4. Use ``monkeypatch.setenv("PATH", value, prepend=os.pathsep)`` to modify ``$PATH``, and
:py:meth:`monkeypatch.chdir <MonkeyPatch.chdir>` to change the context of the current working directory
during a test.

5. Use :py:meth:`monkeypatch.syspath_prepend <MonkeyPatch.syspath_prepend>` to modify ``sys.path`` which will also
call ``pkg_resources.fixup_namespace_packages`` and :py:func:`importlib.invalidate_caches`.

6. Use :py:meth:`monkeypatch.context <MonkeyPatch.context>` to apply patches only in a specific scope, which can help
control teardown of complex fixtures or patches to the stdlib.

See the `monkeypatch blog post`_ for some introduction material
and a discussion of its motivation.

.. _`monkeypatch blog post`: https://tetamap.wordpress.com//2009/03/03/monkeypatching-in-unit-tests-done-right/

Monkeypatching functions
------------------------

Consider a scenario where you are working with user directories. In the context of
testing, you do not want your test to depend on the running user. ``monkeypatch``
can be used to patch functions dependent on the user to always return a
specific value.

In this example, :py:meth:`monkeypatch.setattr <MonkeyPatch.setattr>` is used to patch ``Path.home``
so that the known testing path ``Path("/abc")`` is always used when the test is run.
This removes any dependency on the running user for testing purposes.
:py:meth:`monkeypatch.setattr <MonkeyPatch.setattr>` must be called before the function which will use
the patched function is called.
After the test function finishes the ``Path.home`` modification will be undone.

.. code-block:: python

    # contents of test_module.py with source code and the test
    from pathlib import Path


    def getssh():
        """Simple function to return expanded homedir ssh path."""
        return Path.home() / ".ssh"


    def test_getssh(monkeypatch):
        # mocked return function to replace Path.home
        # always return '/abc'
        def mockreturn():
            return Path("/abc")

        # Application of the monkeypatch to replace Path.home
        # with the behavior of mockreturn defined above.
        monkeypatch.setattr(Path, "home", mockreturn)

        # Calling getssh() will use mockreturn in place of Path.home
        # for this test with the monkeypatch.
        x = getssh()
        assert x == Path("/abc/.ssh")

Monkeypatching returned objects: building mock classes
------------------------------------------------------

:py:meth:`monkeypatch.setattr <MonkeyPatch.setattr>` can be used in conjunction with classes to mock returned
objects from functions instead of values.
Imagine a simple function to take an API url and return the json response.

.. code-block:: python

    # contents of app.py, a simple API retrieval example
    import requests


    def get_json(url):
        """Takes a URL, and returns the JSON."""
        r = requests.get(url)
        return r.json()

We need to mock ``r``, the returned response object for testing purposes.
The mock of ``r`` needs a ``.json()`` method which returns a dictionary.
This can be done in our test file by defining a class to represent ``r``.

.. code-block:: python

    # contents of test_app.py, a simple test for our API retrieval
    # import requests for the purposes of monkeypatching
    import requests

    # our app.py that includes the get_json() function
    # this is the previous code block example
    import app


    # custom class to be the mock return value
    # will override the requests.Response returned from requests.get
    class MockResponse:
        # mock json() method always returns a specific testing dictionary
        @staticmethod
        def json():
            return {"mock_key": "mock_response"}


    def test_get_json(monkeypatch):
        # Any arguments may be passed and mock_get() will always return our
        # mocked object, which only has the .json() method.
        def mock_get(*args, **kwargs):
            return MockResponse()

        # apply the monkeypatch for requests.get to mock_get
        monkeypatch.setattr(requests, "get", mock_get)

        # app.get_json, which contains requests.get, uses the monkeypatch
        result = app.get_json("https://fakeurl")
        assert result["mock_key"] == "mock_response"


``monkeypatch`` applies the mock for ``requests.get`` with our ``mock_get`` function.
The ``mock_get`` function returns an instance of the ``MockResponse`` class, which
has a ``json()`` method defined to return a known testing dictionary and does not
require any outside API connection.

You can build the ``MockResponse`` class with the appropriate degree of complexity for
the scenario you are testing. For instance, it could include an ``ok`` property that
always returns ``True``, or return different values from the ``json()`` mocked method
based on input strings.

This mock can be shared across tests using a ``fixture``:

.. code-block:: python

    # contents of test_app.py, a simple test for our API retrieval
    import pytest
    import requests

    # app.py that includes the get_json() function
    import app


    # custom class to be the mock return value of requests.get()
    class MockResponse:
        @staticmethod
        def json():
            return {"mock_key": "mock_response"}


    # monkeypatched requests.get moved to a fixture
    @pytest.fixture
    def mock_response(monkeypatch):
        """Requests.get() mocked to return {'mock_key':'mock_response'}."""

        def mock_get(*args, **kwargs):
            return MockResponse()

        monkeypatch.setattr(requests, "get", mock_get)


    # notice our test uses the custom fixture instead of monkeypatch directly
    def test_get_json(mock_response):
        result = app.get_json("https://fakeurl")
        assert result["mock_key"] == "mock_response"


Furthermore, if the mock was designed to be applied to all tests, the ``fixture`` could
be moved to a ``conftest.py`` file and use the with ``autouse=True`` option.


Global patch example: preventing "requests" from remote operations
------------------------------------------------------------------

If you want to prevent the "requests" library from performing http
requests in all your tests, you can do:

.. code-block:: python

    # contents of conftest.py
    import pytest


    @pytest.fixture(autouse=True)
    def no_requests(monkeypatch):
        """Remove requests.sessions.Session.request for all tests."""
        monkeypatch.delattr("requests.sessions.Session.request")

This autouse fixture will be executed for each test function and it
will delete the method ``request.session.Session.request``
so that any attempts within tests to create http requests will fail.


.. note::

    Be advised that it is not recommended to patch builtin functions such as ``open``,
    ``compile``, etc., because it might break pytest's internals. If that's
    unavoidable, passing :option:`--tb=native`, :option:`--assert=plain` and :option:`--capture=no` might
    help although there's no guarantee.

.. note::

    Mind that patching ``stdlib`` functions and some third-party libraries used by pytest
    might break pytest itself. Prefer patching the reference that your code uses
    instead of patching the original object in the standard library. For example,
    if your module does ``from os import getcwd``, patch ``mymodule.getcwd``
    rather than ``os.getcwd``.

    For code that you control, a safer long-term pattern is to make dependencies
    explicit so they can be passed into the code under test instead of patched
    globally. When patching a stdlib object is unavoidable, use
    :meth:`MonkeyPatch.context` to limit the patching to the block you want tested:

    .. code-block:: python

        import functools


        def test_partial(monkeypatch):
            with monkeypatch.context() as m:
                m.setattr(functools, "partial", 3)
                assert functools.partial == 3

    See :issue:`3290` for details.


Monkeypatching environment variables
------------------------------------

If you are working with environment variables you often need to safely change the values
or delete them from the system for testing purposes. ``monkeypatch`` provides a mechanism
to do this using the ``setenv`` and ``delenv`` method. Our example code to test:

.. code-block:: python

    # contents of our original code file e.g. code.py
    import os


    def get_os_user_lower():
        """Simple retrieval function.
        Returns lowercase USER or raises OSError."""
        username = os.getenv("USER")

        if username is None:
            raise OSError("USER environment is not set.")

        return username.lower()

There are two potential paths. First, the ``USER`` environment variable is set to a
value. Second, the ``USER`` environment variable does not exist. Using ``monkeypatch``
both paths can be safely tested without impacting the running environment:

.. code-block:: python

    # contents of our test file e.g. test_code.py
    import pytest


    def test_upper_to_lower(monkeypatch):
        """Set the USER env var to assert the behavior."""
        monkeypatch.setenv("USER", "TestingUser")
        assert get_os_user_lower() == "testinguser"


    def test_raise_exception(monkeypatch):
        """Remove the USER env var and assert OSError is raised."""
        monkeypatch.delenv("USER", raising=False)

        with pytest.raises(OSError):
            _ = get_os_user_lower()

This behavior can be moved into ``fixture`` structures and shared across tests:

.. code-block:: python

    # contents of our test file e.g. test_code.py
    import pytest


    @pytest.fixture
    def mock_env_user(monkeypatch):
        monkeypatch.setenv("USER", "TestingUser")


    @pytest.fixture
    def mock_env_missing(monkeypatch):
        monkeypatch.delenv("USER", raising=False)


    # notice the tests reference the fixtures for mocks
    def test_upper_to_lower(mock_env_user):
        assert get_os_user_lower() == "testinguser"


    def test_raise_exception(mock_env_missing):
        with pytest.raises(OSError):
            _ = get_os_user_lower()


Monkeypatching dictionaries
---------------------------

:py:meth:`monkeypatch.setitem <MonkeyPatch.setitem>` can be used to safely set the values of dictionaries
to specific values during tests. Take this simplified connection string example:

.. code-block:: python

    # contents of app.py to generate a simple connection string
    DEFAULT_CONFIG = {"user": "user1", "database": "db1"}


    def create_connection_string(config=None):
        """Creates a connection string from input or defaults."""
        config = config or DEFAULT_CONFIG
        return f"User Id={config['user']}; Location={config['database']};"

For testing purposes we can patch the ``DEFAULT_CONFIG`` dictionary to specific values.

.. code-block:: python

    # contents of test_app.py
    # app.py with the connection string function (prior code block)
    import app


    def test_connection(monkeypatch):
        # Patch the values of DEFAULT_CONFIG to specific
        # testing values only for this test.
        monkeypatch.setitem(app.DEFAULT_CONFIG, "user", "test_user")
        monkeypatch.setitem(app.DEFAULT_CONFIG, "database", "test_db")

        # expected result based on the mocks
        expected = "User Id=test_user; Location=test_db;"

        # the test uses the monkeypatched dictionary settings
        result = app.create_connection_string()
        assert result == expected

You can use the :py:meth:`monkeypatch.delitem <MonkeyPatch.delitem>` to remove values.

.. code-block:: python

    # contents of test_app.py
    import pytest

    # app.py with the connection string function
    import app


    def test_missing_user(monkeypatch):
        # patch the DEFAULT_CONFIG to be missing the 'user' key
        monkeypatch.delitem(app.DEFAULT_CONFIG, "user", raising=False)

        # Key error expected because a config is not passed, and the
        # default is now missing the 'user' entry.
        with pytest.raises(KeyError):
            _ = app.create_connection_string()


The modularity of fixtures gives you the flexibility to define
separate fixtures for each potential mock and reference them in the needed tests.

.. code-block:: python

    # contents of test_app.py
    import pytest

    # app.py with the connection string function
    import app


    # all of the mocks are moved into separated fixtures
    @pytest.fixture
    def mock_test_user(monkeypatch):
        """Set the DEFAULT_CONFIG user to test_user."""
        monkeypatch.setitem(app.DEFAULT_CONFIG, "user", "test_user")


    @pytest.fixture
    def mock_test_database(monkeypatch):
        """Set the DEFAULT_CONFIG database to test_db."""
        monkeypatch.setitem(app.DEFAULT_CONFIG, "database", "test_db")


    @pytest.fixture
    def mock_missing_default_user(monkeypatch):
        """Remove the user key from DEFAULT_CONFIG"""
        monkeypatch.delitem(app.DEFAULT_CONFIG, "user", raising=False)


    # tests reference only the fixture mocks that are needed
    def test_connection(mock_test_user, mock_test_database):
        expected = "User Id=test_user; Location=test_db;"

        result = app.create_connection_string()
        assert result == expected


    def test_missing_user(mock_missing_default_user):
        with pytest.raises(KeyError):
            _ = app.create_connection_string()


.. currentmodule:: pytest

API Reference
-------------

Consult the docs for the :class:`MonkeyPatch` class.
