Automatic Buildout Updates
==========================

When a buildout is run, one of the first steps performed is to check for
updates to either zc.buildout or setuptools.  To
demonstrate this, we've created some "new releases" of buildout and
setuptools in a new_releases folder:

    >>> ls(new_releases)
    -  zc_buildout-91.0-py3-none-any.whl
    -  zc_buildout-NINETYNINE.NINETYNINE-py3-none-any.whl

Let's update the sample buildout.cfg to look in this area:

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... find-links = %(new_releases)s
    ... index = %(new_releases)s
    ... parts = show-versions
    ... develop = showversions
    ...
    ... [show-versions]
    ... recipe = showversions
    ... """ % dict(new_releases=new_releases))

We'll also include a recipe that echos the versions of setuptools and
zc.buildout used:

    >>> mkdir(sample_buildout, 'showversions')

    >>> write(sample_buildout, 'showversions', 'showversions.py',
    ... """
    ... import pkg_resources
    ... import sys
    ... print_ = lambda *a: sys.stdout.write(' '.join(map(str, a))+'\\n')
    ...
    ... class Recipe:
    ...
    ...     def __init__(self, buildout, name, options):
    ...         pass
    ...
    ...     def install(self):
    ...         for project in ['zc.buildout']:
    ...             req = pkg_resources.Requirement.parse(project)
    ...             print_(project, pkg_resources.working_set.find(req).version)
    ...         return ()
    ...     update = install
    ... """)


    >>> write(sample_buildout, 'showversions', 'setup.py',
    ... """
    ... from setuptools import setup
    ...
    ... setup(
    ...     name = "showversions",
    ...     entry_points = {'zc.buildout': ['default = showversions:Recipe']},
    ...     )
    ... """)

The installed zc.buildout version is in development mode, so it won't upgrade itself.

    >>> print_(system(buildout), end='')
    Develop:...
    Installing show-versions.
    zc.buildout V.V

But if we run the buildout and tell it to use version 50, the buildout will upgrade itself:

    >>> print_(system(buildout + ' versions:zc.buildout=91.0'), end='')
    Getting distribution for 'zc.buildout==91.0'.
    Got zc.buildout V.V
    Upgraded:
      zc.buildout V.V
    Restarting.
    Generated script '/sample-buildout/bin/buildout'.
    Develop: '/sample-buildout/showversions'
    Updating show-versions.
    zc.buildout V.V

Now if we run the buildout without explicit version, the buildout will upgrade itself to the
newest version found in new releases:

    >>> print_(system(buildout), end='')
    Got zc.buildout NINETYNINE.NINETYNINE.
    Upgraded:
      zc.buildout version NINETYNINE.NINETYNINE;
    Restarting.
    Generated script '/sample-buildout/bin/buildout'.
    Develop: '/sample-buildout/showversions'
    Updating show-versions.
    zc.buildout NINETYNINE.NINETYNINE

Our buildout script has been updated to use the new eggs:

    >>> cat(sample_buildout, 'bin', 'buildout')
    #!/usr/local/bin/python2.7
    <BLANKLINE>
    import sys
    sys.path[0:0] = [
      '/sample-buildout/eggs/v5/zc.buildout-NINETYNINE.NINETYNINE-pyN.N.egg',
    ...
      ]
    <BLANKLINE>
    import zc.buildout.buildout
    <BLANKLINE>
    if __name__ == '__main__':
        sys.exit(zc.buildout.buildout.main())

Now, let's recreate the sample buildout. If we specify constraints on
the versions of zc.buildout and setuptools to use, running the buildout
will install earlier versions of these packages:

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... find-links = %(new_releases)s
    ... index = %(new_releases)s
    ... parts = show-versions
    ... develop = showversions
    ...
    ... [versions]
    ... zc.buildout = < 99
    ...
    ... [show-versions]
    ... recipe = showversions
    ... """ % dict(new_releases=new_releases))

Now we can see that we actually "upgrade" to an earlier version.

    >>> print_(system(buildout), end='')
    Upgraded:
      zc.buildout V.V
    Restarting.
    Generated script '/sample-buildout/bin/buildout'.
    Develop: '/sample-buildout/showversions'
    Updating show-versions.
    zc.buildout V.V

There are a number of cases, described below, in which the updates
don't happen.

We won't upgrade in offline mode:

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... find-links = %(new_releases)s
    ... index = %(new_releases)s
    ... parts = show-versions
    ... develop = showversions
    ...
    ... [show-versions]
    ... recipe = showversions
    ... """ % dict(new_releases=new_releases))

    >>> print_(system(buildout+' -o'), end='')
    Develop: '/sample-buildout/showversions'
    Updating show-versions.
    zc.buildout 1.0.0

Or in non-newest mode:

    >>> print_(system(buildout+' -N'), end='')
    Develop: '/sample-buildout/showversions'
    Updating show-versions.
    zc.buildout 1.0.0

We also won't upgrade if the buildout script being run isn't in the
buildouts bin directory.  To see this we'll create a new buildout
directory:

    >>> sample_buildout2 = tmpdir('sample_buildout2')
    >>> write(sample_buildout2, 'buildout.cfg',
    ... """
    ... [buildout]
    ... find-links = %(new_releases)s
    ... index = %(new_releases)s
    ... parts =
    ...
    ... [versions]
    ... zc.buildout = 99.99
    ... """ % dict(new_releases=new_releases))

    >>> cd(sample_buildout2)
    >>> print_(system(buildout), end='')
    Creating directory '/sample_buildout2/eggs/v5'.
    Creating directory '/sample_buildout2/bin'.
    Creating directory '/sample_buildout2/parts'.
    Creating directory '/sample_buildout2/develop-eggs'.
    Getting distribution for 'zc.buildout==NINETYNINE.NINETYNINE'.
    Got zc.buildout NINETYNINE.NINETYNINE.
    Not upgrading because not running a local buildout command.

    >>> ls('bin')

.. The relative-paths option is honored:

    >>> cd(sample_buildout)
    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... find-links = %(new_releases)s
    ... index = %(new_releases)s
    ... parts = show-versions
    ... develop = showversions
    ... relative-paths = true
    ...
    ... [show-versions]
    ... recipe = showversions
    ... """ % dict(new_releases=new_releases))

    >>> print_(system(buildout), end='')
    Upgraded:
      zc.buildout version NINETYNINE.NINETYNINE;
    Restarting.
    Generated script '/sample-buildout/bin/buildout'.
    Develop: '/sample-buildout/showversions'
    Section `buildout` contains unused option(s): 'relative-paths'.
    This may be an indication for either a typo in the option's name or a bug in the used recipe.
    Updating show-versions.
    zc.buildout NINETYNINE.NINETYNINE

    >>> cat('bin', 'buildout') # doctest +ELL
    #!/usr/local/bin/python2.7
    <BLANKLINE>
    import os
    <BLANKLINE>
    join = os.path.join
    base = os.path.dirname(os.path.abspath(os.path.realpath(__file__)))
    base = os.path.dirname(base)
    <BLANKLINE>
    import sys
    sys.path[0:0] = [
      join(base, 'eggs/v5/zc.buildout-NINETYNINE.NINETYNINE-pyN.N.egg'),
    ...
      ]
    <BLANKLINE>
    import zc.buildout.buildout
    <BLANKLINE>
    if __name__ == '__main__':
        sys.exit(zc.buildout.buildout.main())

When buildout restarts and the restarted buildout exits with an error code,
the original buildout that called the second buildout also exits with that
error code. Otherwise build scripts can erroneously detect a successful
buildout run even if it failed.

Make a recipe that fails:

    >>> mkdir(sample_buildout, 'failrecipe')
    >>> write(sample_buildout, 'failrecipe', 'failrecipe.py',
    ... """
    ... import pkg_resources
    ... import sys
    ... print_ = lambda *a: sys.stdout.write(' '.join(map(str, a))+'\\n')
    ...
    ... class Recipe:
    ...
    ...     def __init__(self, buildout, name, options):
    ...         sys.exit('recipe sys-exits')
    ...
    ...     def install(self):
    ...         pass
    ...
    ...     update = install
    ... """)
    >>> write(sample_buildout, 'failrecipe', 'setup.py',
    ... """
    ... from setuptools import setup
    ...
    ... setup(
    ...     name = "failrecipe",
    ...     entry_points = {'zc.buildout': ['default = failrecipe:Recipe']},
    ...     )
    ... """)

Let's downgrade again, triggering a restart. And use the failing recipe that
gives us a sys.exit:

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... find-links = %(new_releases)s
    ... index = %(new_releases)s
    ... parts = fail
    ... develop = failrecipe
    ...
    ... [versions]
    ... zc.buildout = < 99
    ...
    ... [fail]
    ... recipe = failrecipe
    ... """ % dict(new_releases=new_releases))

Run the buildout:

    >>> print_(system(buildout, with_exit_code=True), end='')
    Upgraded:
      zc.buildout V.V
    Restarting.
    Generated script '/sample-buildout/bin/buildout'.
    Develop: '/sample-buildout/failrecipe'
    recipe sys-exits
    EXIT CODE: 1
