jepler/virtualenv
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Stat rewrite - POC

Browse source 
Signed-off-by: Bernat Gabor <bgabor8@bloomberg.net>
This commit is contained in:
Bernát Gábor 2020-01-10 15:35:38 +00:00 committed by Bernat Gabor
commit d47ab61dc2
No known key found for this signature in database
GPG key ID: 9A12C876AC23ED79
97 changed files with 7035 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

27
.coveragerc Normal file
View file

@ -0,0 +1,27 @@
[coverage:report]
skip_covered = False
show_missing = True
exclude_lines =
\#\s*pragma: no cover
^\s*raise AssertionError\b
^\s*raise NotImplementedError\b
^\s*raise$
^if __name__ == ['"]__main__['"]:$
omit =
# site.py is ran before the coverage can be enabled, no way to measure coverage on this
src/virtualenv/interpreters/create/impl/cpython/site.py
[coverage:paths]
source =
src
.tox/*/lib/python*/site-packages
.tox/pypy*/site-packages
.tox\*\Lib\site-packages\
*/src
*\src
[coverage:run]
branch = false
parallel = true
source =
${_COVERAGE_SRC}

1
.gitattributes vendored Normal file
View file

@ -0,0 +1 @@
*.bat text eol=crlf

13
.github/ISSUE_TEMPLATE.md vendored Normal file
View file

@ -0,0 +1,13 @@
Thanks for submitting an issue!
If submitting a BUG please provide:
- [ ] Minimal reproducible example or detailed descriptions
- [ ] OS and `pip list` output
if submitting an ENHANCEMENT issue:
- [ ] a clear problem statement with an example
- [ ] suggested change with example
- [ ] if you have want help to do a PR yourself

6
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View file

@ -0,0 +1,6 @@
## Thanks for contributing a pull request, see checklist all is good!
- [ ] wrote descriptive pull request text
- [ ] added/updated test(s)
- [ ] updated/extended the documentation
- [ ] added news fragment in ``docs/changelog`` folder

2
.github/config.yml vendored Normal file
View file

@ -0,0 +1,2 @@
rtd:
project: virtualenv

11
.github/stale.yml vendored Normal file
View file

@ -0,0 +1,11 @@
daysUntilStale: 90
daysUntilClose: 7
exemptLabels:
- pinned
- security
staleLabel: wontfix
markComment: >
This issue has been automatically marked as stale because it has not had
recent activity. It will be closed if no further activity occurs. Just add a comment
if you want to keep it open. Thank you for your contributions.
closeComment: false

25
.gitignore vendored Normal file
View file

@ -0,0 +1,25 @@
# packaging
virtualenv.egg-info
build
dist
*.egg
.eggs
# python
*.py[cod]
*$py.class
# tools
.tox
.*_cache
.DS_Store
# IDE
.idea
.vscode
/docs/_draft.rst
/pip-wheel-metadata
/src/virtualenv/version.py
/src/virtualenv/out
/*env*

43
.pre-commit-config.yaml Normal file
View file

@ -0,0 +1,43 @@
repos:
- repo: https://github.com/ambv/black
rev: 19.10b0
hooks:
- id: black
args: [--safe]
language_version: python3.8
- repo: https://github.com/asottile/blacken-docs
rev: v1.3.0
hooks:
- id: blacken-docs
additional_dependencies: [black==19.3b0]
language_version: python3.8
- repo: https://github.com/asottile/seed-isort-config
rev: v1.9.3
hooks:
- id: seed-isort-config
args: [--application-directories, '.:src']
- repo: https://github.com/pre-commit/mirrors-isort
rev: v4.3.21
hooks:
- id: isort
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.4.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: debug-statements
- id: check-merge-conflict
- id: trailing-whitespace
- id: check-docstring-first
- id: flake8
additional_dependencies: ["flake8-bugbear == 19.8.0"]
language_version: python3.8
- repo: https://github.com/asottile/pyupgrade
rev: v1.25.1
hooks:
- id: pyupgrade
- repo: https://github.com/pre-commit/pygrep-hooks
rev: v1.4.2
hooks:
- id: rst-backticks

93
AUTHORS.txt Normal file
View file

@ -0,0 +1,93 @@
Author
------
Ian Bicking
Maintainers
-----------
Brian Rosner
Bernat Gabor
Carl Meyer
Jannis Leidel
Paul Moore
Paul Nasrat
Marcus Smith
Contributors
------------
Alex Grönholm
Anatoly Techtonik
Antonio Cuni
Antonio Valentino
Armin Ronacher
Barry Warsaw
Benjamin Root
Bradley Ayers
Branden Rolston
Brandon Carl
Brian Kearns
Cap Petschulat
CBWhiz
Chris Adams
Chris McDonough
Christos Kontas
Christian Hudon
Christian Stefanescu
Christopher Nilsson
Cliff Xuan
Curt Micol
Damien Nozay
Dan Sully
Daniel Hahler
Daniel Holth
David Schoonover
Denis Costa
Doug Hellmann
Doug Napoleone
Douglas Creager
Eduard-Cristian Stefan
Erik M. Bray
Ethan Jucovy
Gabriel de Perthuis
Gunnlaugur Thor Briem
Graham Dennis
Greg Haskins
Jason Penney
Jason R. Coombs
Jeff Hammel
Jeremy Orem
Jason Penney
Jason R. Coombs
John Kleint
Jonathan Griffin
Jonathan Hitchcock
Jorge Vargas
Josh Bronson
Kamil Kisiel
Kyle Gibson
Konstantin Zemlyak
Kumar McMillan
Lars Francke
Marc Abramowitz
Mika Laitio
Mike Hommey
Miki Tebeka
Philip Jenvey
Philippe Ombredanne
Piotr Dobrogost
Preston Holmes
Ralf Schmitt
Raul Leal
Ronny Pfannschmidt
Satrajit Ghosh
Sergio de Carvalho
Stefano Rivera
Tarek Ziadé
Thomas Aglassinger
Vinay Sajip
Vitaly Babiy
Vladimir Rutsky
Wang Xuerui
Wouter De Borger

25
CONTRIBUTING.rst Normal file
View file

@ -0,0 +1,25 @@
virtualenv
==========
See docs/index.rst for user documentation.
Contributor notes
-----------------
* virtualenv is designed to work on python 2 and 3 with a single code base.
Use Python 3 print-function syntax, and always ``use sys.exc_info()[1]``
inside the ``except`` block to get at exception objects.
* Pull requests should be made against ``master`` branch, which is also our
latest stable version.
* All changes to files inside virtualenv_embedded must be integrated to
``virtualenv.py`` with ``tox -e embed``. The tox run will report failure
when changes are integrated, as a flag for CI.
* The codebase must be linted with ``tox -e fix_lint`` before being merged.
The tox run will report failure when the linters revise code, as a flag
for CI.
.. _git-flow: https://github.com/nvie/gitflow
.. _coordinate development: http://nvie.com/posts/a-successful-git-branching-model/

22
LICENSE.txt Normal file
View file

@ -0,0 +1,22 @@
Copyright (c) 2007 Ian Bicking and Contributors
Copyright (c) 2009 Ian Bicking, The Open Planning Project
Copyright (c) 2011-2016 The virtualenv developers
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

12
MANIFEST.in Normal file
View file

@ -0,0 +1,12 @@
# setuptools-scm by default adds all SCM tracked files, we prune the following maintenance related ones (sdist only)
exclude .gitattributes
exclude .gitignore
exclude .github/*
exclude azure-pipelines.yml
exclude CONTRIBUTING.rst
exclude readthedocs.yml
exclude MANIFEST.in
exclude tasks/release.py
exclude tasks/upgrade_wheels.py

44
README.rst Normal file
View file

@ -0,0 +1,44 @@
virtualenv
==========
A tool for creating isolated 'virtual' python environments.
.. image:: https://img.shields.io/pypi/v/virtualenv.svg
:target: https://pypi.org/project/virtualenv
:alt: Latest version on PyPi
.. image:: https://img.shields.io/pypi/pyversions/virtualenv.svg
:target: https://pypi.org/project/virtualenv/
:alt: Supported Python versions
.. image:: https://dev.azure.com/pypa/virtualenv/_apis/build/status/pypa.virtualenv?branchName=master
:target: https://dev.azure.com/pypa/virtualenv/_build/latest?definitionId=11&branchName=master
:alt: Azure Pipelines build status
.. image:: https://readthedocs.org/projects/virtualenv/badge/?version=latest&style=flat-square
:target: https://virtualenv.readthedocs.io/en/latest/?badge=latest
:alt: Documentation status
.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
:target: https://github.com/ambv/black
:alt: Code style: black
.. image:: https://pepy.tech/badge/virtualenv/month
:target: https://pepy.tech/project/virtualenv/month
:alt: Downloads
* `Installation <https://virtualenv.pypa.io/en/latest/installation.html>`_
* `Documentation <https://virtualenv.pypa.io/>`_
* `Changelog <https://virtualenv.pypa.io/en/latest/changes.html>`_
* `Issues <https://github.com/pypa/virtualenv/issues>`_
* `PyPI <https://pypi.org/project/virtualenv/>`_
* `Github <https://github.com/pypa/virtualenv>`_
* `User mailing list <http://groups.google.com/group/python-virtualenv>`_
* `Dev mailing list <http://groups.google.com/group/pypa-dev>`_
* User IRC: `#pypa on Freenode <https://webchat.freenode.net/?channels=%23pypa>`_
* Dev IRC: `#pypa-dev on Freenode <https://webchat.freenode.net/?channels=%23pypa-dev>`_
Code of Conduct
---------------
Everyone interacting in the virtualenv project's codebases, issue trackers,
chat rooms, and mailing lists is expected to follow the
`PyPA Code of Conduct`_.
.. _PyPA Code of Conduct: https://www.pypa.io/en/latest/code-of-conduct/

75
azure-pipelines.yml Normal file
View file

@ -0,0 +1,75 @@
name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr)
resources:
repositories:
- repository: tox
type: github
endpoint: github-gb
name: tox-dev/azure-pipelines-template
ref: master
trigger:
batch: true
branches:
include:
- master
- rewrite
- refs/tags/*
pr:
branches:
include:
- '*'
schedules:
- cron: "12 0 * * *"
displayName: Daily build
branches:
include: [ master, rewrite ]
always: true
variables:
PYTEST_ADDOPTS: "-v -v -ra --showlocals --durations=15"
PYTEST_XDIST_PROC_NR: 'auto'
CI_RUN: 'yes'
UPGRADE_ADVISORY: 'yes'
jobs:
- template: run-tox-env.yml@tox
parameters:
jobs:
py38:
image: [linux, windows, macOs]
py37:
image: [linux, windows, macOs]
py36:
image: [linux, windows, macOs]
py35:
image: [linux, windows, macOs]
py27:
image: [linux, windows, macOs]
fix_lint:
image: [linux, windows]
docs:
image: [linux, windows]
package_readme:
image: [linux, windows]
upgrade:
image: [linux, windows]
dev: null
before:
- script: 'sudo apt-get update -y && sudo apt-get install fish csh'
condition: and(succeeded(), eq(variables['image_name'], 'linux'), in(variables['TOXENV'], 'py38', 'py37', 'py36', 'py35', 'py34', 'py27'))
displayName: install fish and csh via apt-get
- script: 'brew update -vvv && brew install fish tcsh'
condition: and(succeeded(), eq(variables['image_name'], 'macOs'), in(variables['TOXENV'], 'py38', 'py37', 'py36', 'py35', 'py34', 'py27'))
displayName: install fish and csh via brew
coverage:
with_toxenv: 'coverage' # generate .tox/.coverage, .tox/coverage.xml after test run
for_envs: [py38, py37, py36, py35, py27]
- ${{ if startsWith(variables['Build.SourceBranch'], 'refs/tags/') }}:
- template: publish-pypi.yml@tox
parameters:
external_feed: 'gb'
pypi_remote: 'pypi-gb'
dependsOn: [fix_lint, embed, cross_python3, cross_python3, docs, report_coverage, dev, package_readme]

0
docs/changelog/.gitkeep Normal file
View file

18
docs/changelog/examples.rst Normal file
View file

@ -0,0 +1,18 @@
.. examples for changelog entries adding to your Pull Requests
file ``544.doc.rst``::
explain everything much better - by ``passionate_technicalwriter``
file ``544.feature.rst``::
``tox --version`` now shows information about all registered plugins - by ``obestwalter``
file ``571.bugfix.rst``::
``skip_install`` overrides ``usedevelop`` (``usedevelop`` is an option to choose the
installation type if the package is installed and ``skip_install`` determines if it should be
installed at all) - by ``ferdonline``
.. see pyproject.toml for all available categories

31
docs/changelog/template.jinja2 Normal file
View file

@ -0,0 +1,31 @@
{% for section, _ in sections.items() %}
{% set underline = underlines[0] %}
{% if section %}
{{section}}
{{ underline * section|length }}
{% set underline = underlines[1] %}
{% endif %}
{% if sections[section] %}
{% for category, val in definitions.items() if category in sections[section] %}
{{ definitions[category]['name'] }}
{{ underline * definitions[category]['name']|length }}
{% if definitions[category]['showcontent'] %}
{% for text, values in sections[section][category].items() %}
- {{ text }} ({{ values|join(', ') }})
{% endfor %}
{% else %}
- {{ sections[section][category]['']|join(', ') }}
{% endif %}
{% if sections[section][category]|length == 0 %}
No significant changes.
{% endif %}
{% endfor %}
{% else %}
No significant changes.
{% endif %}
{% endfor %}

1325
docs/changes.rst Normal file
View file

File diff suppressed because it is too large Load diff

74
docs/conf.py Normal file
View file

@ -0,0 +1,74 @@
from __future__ import absolute_import, unicode_literals
import os
import re
import subprocess
import sys
from pathlib import Path
from virtualenv import __version__
extensions = ["sphinx.ext.autodoc", "sphinx.ext.extlinks"]
source_suffix = ".rst"
master_doc = "index"
project = "virtualenv"
# noinspection PyShadowingBuiltins
copyright = "2007-2018, Ian Bicking, The Open Planning Project, PyPA"
ROOT_SRC_TREE_DIR = Path(__file__).parents[1]
def generate_draft_news():
home = "https://github.com"
issue = "{}/issue".format(home)
fragments_path = ROOT_SRC_TREE_DIR / "docs" / "changelog"
for pattern, replacement in (
(r"[^`]@([^,\s]+)", r"`@\1 <{}/\1>`_".format(home)),
(r"[^`]#([\d]+)", r"`#pr\1 <{}/\1>`_".format(issue)),
):
for path in fragments_path.glob("*.rst"):
path.write_text(re.sub(pattern, replacement, path.read_text()))
env = os.environ.copy()
env["PATH"] += os.pathsep.join([os.path.dirname(sys.executable)] + env["PATH"].split(os.pathsep))
changelog = subprocess.check_output(
["towncrier", "--draft", "--version", "DRAFT"], cwd=str(ROOT_SRC_TREE_DIR), env=env
).decode("utf-8")
if "No significant changes" in changelog:
content = ""
else:
note = "*Changes in master, but not released yet are under the draft section*."
content = "{}\n\n{}".format(note, changelog)
(ROOT_SRC_TREE_DIR / "docs" / "_draft.rst").write_text(content)
generate_draft_news()
version = ".".join(__version__.split(".")[:2])
release = __version__
today_fmt = "%B %d, %Y"
unused_docs = []
pygments_style = "sphinx"
exclude_patterns = ["changelog/*"]
extlinks = {
"issue": ("https://github.com/pypa/virtualenv/issues/%s", "#"),
"pull": ("https://github.com/pypa/virtualenv/pull/%s", "PR #"),
}
html_theme = "sphinx_rtd_theme"
html_theme_options = {
"canonical_url": "https://virtualenv.pypa.io/en/latest/",
"logo_only": False,
"display_version": True,
"prev_next_buttons_location": "bottom",
"style_external_links": True,
# Toc options
"collapse_navigation": True,
"sticky_navigation": True,
"navigation_depth": 4,
"includehidden": True,
"titles_only": False,
}
html_last_updated_fmt = "%b %d, %Y"
htmlhelp_basename = "Pastedoc"

56
docs/development.rst Normal file
View file

@ -0,0 +1,56 @@
Development
===========
Contributing
------------
Refer to the `pip development`_ documentation - it applies equally to
virtualenv, except that virtualenv issues should be filed on the `virtualenv
repo`_ at GitHub.
Virtualenv's release schedule is tied to pip's -- each time there's a new pip
release, there will be a new virtualenv release that bundles the new version of
pip.
Files in the ``virtualenv_embedded/`` subdirectory are embedded into
``virtualenv.py`` itself as base64-encoded strings (in order to support
single-file use of ``virtualenv.py`` without installing it). If your patch
changes any file in ``virtualenv_embedded/``, run ``tox -e embed`` to update
the embedded version of that file in ``virtualenv.py``; commit that and submit
it as part of your patch / pull request. The tox run will report failure
when changes are embedded, as a flag for CI.
The codebase should be linted before a pull request is merged by running
``tox -e fix_lint``. The tox run will report failure when any linting
revisions are required, as a flag for CI.
.. _pip development: https://pip.pypa.io/en/latest/development/
.. _virtualenv repo: https://github.com/pypa/virtualenv/
Running the tests
-----------------
The easy way to run tests (handles test dependencies automatically, works with the ``sdist`` too)::
$ tox
Note you need to first install tox separately by using::
$ python -m pip --user install -U tox
Run ``python -m tox -av`` for a list of all supported Python environments or just run the
tests in all of the available ones by running just ``tox``.
Status and License
------------------
``virtualenv`` is a successor to `workingenv
<http://cheeseshop.python.org/pypi/workingenv.py>`_, and an extension
of `virtual-python
<http://peak.telecommunity.com/DevCenter/EasyInstall#creating-a-virtual-python>`_.
It was written by Ian Bicking, sponsored by the `Open Planning
Project <http://openplans.org>`_ and is now maintained by a
`group of developers <https://github.com/pypa/virtualenv/raw/master/AUTHORS.txt>`_.
It is licensed under an
`MIT-style permissive license <https://github.com/pypa/virtualenv/raw/master/LICENSE.txt>`_.

138
docs/index.rst Normal file
View file

@ -0,0 +1,138 @@
Virtualenv
==========
`Mailing list <http://groups.google.com/group/python-virtualenv>`_ |
`Issues <https://github.com/pypa/virtualenv/issues>`_ |
`Github <https://github.com/pypa/virtualenv>`_ |
`PyPI <https://pypi.org/project/virtualenv/>`_ |
User IRC: #pypa
Dev IRC: #pypa-dev
Introduction
------------
``virtualenv`` is a tool to create isolated Python environments. Since
Python 3.3, a subset of it has been integrated into the standard
library under the `venv module <https://docs.python.org/3/library/venv.html>`_.
Note though, that the ``venv`` module does not offer all features of this
library (e.g. cannot create bootstrap scripts, cannot create virtual
environments for other python versions than the host python,
not relocatable, etc.). Tools in general as such still may prefer using
virtualenv for its ease of upgrading (via pip), unified handling of different
Python versions and some more advanced features.
The basic problem being addressed is one of dependencies and versions,
and indirectly permissions. Imagine you have an application that
needs version 1 of LibFoo, but another application requires version
2. How can you use both these applications? If you install
everything into ``/usr/lib/python2.7/site-packages`` (or whatever your
platform's standard location is), it's easy to end up in a situation
where you unintentionally upgrade an application that shouldn't be
upgraded.
Or more generally, what if you want to install an application *and
leave it be*? If an application works, any change in its libraries or
the versions of those libraries can break the application.
Also, what if you can't install packages into the global
``site-packages`` directory? For instance, on a shared host.
In all these cases, ``virtualenv`` can help you. It creates an
environment that has its own installation directories, that doesn't
share libraries with other virtualenv environments (and optionally
doesn't access the globally installed libraries either).
.. comment: split here
.. toctree::
:maxdepth: 2
installation
userguide
reference
development
changes
Other Documentation and Links
-----------------------------
* `Blog announcement of virtualenv`__.
.. __: http://blog.ianbicking.org/2007/10/10/workingenv-is-dead-long-live-virtualenv/
* James Gardner has written a tutorial on using `virtualenv with
Pylons
<http://wiki.pylonshq.com/display/pylonscookbook/Using+a+Virtualenv+Sandbox>`_.
* Chris Perkins created a `showmedo video including virtualenv
<http://showmedo.com/videos/video?name=2910000&fromSeriesID=291>`_.
* Doug Hellmann's `virtualenvwrapper`_ is a useful set of scripts to make
your workflow with many virtualenvs even easier. `His initial blog post on it`__.
He also wrote `an example of using virtualenv to try IPython`__.
.. _virtualenvwrapper: https://pypi.org/project/virtualenvwrapper/
.. __: https://doughellmann.com/blog/2008/05/01/virtualenvwrapper/
.. __: https://doughellmann.com/blog/2008/02/01/ipython-and-virtualenv/
* `Pew`_ is another wrapper for virtualenv that makes use of a different
activation technique.
.. _Pew: https://pypi.org/project/pew/
* `Using virtualenv with mod_wsgi
<http://code.google.com/p/modwsgi/wiki/VirtualEnvironments>`_.
* `virtualenv commands
<https://github.com/thisismedium/virtualenv-commands>`_ for some more
workflow-related tools around virtualenv.
* PyCon US 2011 talk: `Reverse-engineering Ian Bicking's brain: inside pip and virtualenv
<http://pyvideo.org/video/568/reverse-engineering-ian-bicking--39-s-brain--insi>`_.
By the end of the talk, you'll have a good idea exactly how pip
and virtualenv do their magic, and where to go looking in the source
for particular behaviors or bug fixes.
Compare & Contrast with Alternatives
------------------------------------
There are several alternatives that create isolated environments:
* Python 3's `venv module <https://docs.python.org/3/library/venv.html>`_
is recommended for projects that no longer need to support Python 2 and want
to create just simple environments for the host python.
* ``workingenv`` (which I do not suggest you use anymore) is the
predecessor to this library. It used the main Python interpreter,
but relied on setting ``$PYTHONPATH`` to activate the environment.
This causes problems when running Python scripts that aren't part of
the environment (e.g., a globally installed ``hg`` or ``bzr``). It
also conflicted a lot with Setuptools.
* `virtual-python
<http://peak.telecommunity.com/DevCenter/EasyInstall#creating-a-virtual-python>`_
is also a predecessor to this library. It uses only symlinks, so it
couldn't work on Windows. It also symlinks over the *entire*
standard library and global ``site-packages``. As a result, it
won't see new additions to the global ``site-packages``.
This script only symlinks a small portion of the standard library
into the environment, and so on Windows it is feasible to simply
copy these files over. Also, it creates a new/empty
``site-packages`` and also adds the global ``site-packages`` to the
path, so updates are tracked separately. This script also installs
Setuptools automatically, saving a step and avoiding the need for
network access.
* `zc.buildout <http://pypi.org/project/zc.buildout>`_ doesn't
create an isolated Python environment in the same style, but
achieves similar results through a declarative config file that sets
up scripts with very particular packages. As a declarative system,
it is somewhat easier to repeat and manage, but more difficult to
experiment with. ``zc.buildout`` includes the ability to setup
non-Python systems (e.g., a database server or an Apache instance).
I *strongly* recommend anyone doing application development or
deployment use one of these tools.

69
docs/installation.rst Normal file
View file

@ -0,0 +1,69 @@
Installation
============
.. warning::
We advise installing virtualenv-1.9 or greater. Prior to version 1.9, the
pip included in virtualenv did not download from PyPI over SSL.
.. warning::
When using pip to install virtualenv, we advise using pip 1.3 or greater.
Prior to version 1.3, pip did not download from PyPI over SSL.
.. warning::
We advise against using easy_install to install virtualenv when using
setuptools < 0.9.7, because easy_install didn't download from PyPI over SSL
and was broken in some subtle ways.
In Windows, run the ``pip`` provided by your Python installation to install ``virtualenv``.
::
> pip install virtualenv
In non-Windows systems it is discouraged to run ``pip`` as root including with ``sudo``.
Generally use your system package manager if it provides a package.
This avoids conflicts in versions and file locations between the system package manager and ``pip``.
See your distribution's package manager documentation for instructions on using it to install ``virtualenv``.
Using ``pip install --user`` is less hazardous but can still cause trouble within the particular user account.
If a system package expects the system provided ``virtualenv`` and an incompatible version is installed with ``--user`` that package may have problems within that user account.
To install within your user account with ``pip`` (if you have pip 1.3 or greater installed):
::
$ pip install --user virtualenv
Note: The specific ``bin`` path may vary per distribution but is often ``~/.local/bin`` and must be added to your ``$PATH`` if not already present.
Or to get the latest unreleased dev version:
::
$ pip install --user https://github.com/pypa/virtualenv/tarball/master
To install version ``X.X.X`` globally from source:
::
$ pip install --user https://github.com/pypa/virtualenv/tarball/X.X.X
To *use* locally from source:
::
$ curl --location --output virtualenv-X.X.X.tar.gz https://github.com/pypa/virtualenv/tarball/X.X.X
$ tar xvfz virtualenv-X.X.X.tar.gz
$ cd pypa-virtualenv-YYYYYY
$ python virtualenv.py myVE
.. note::
The ``virtualenv.py`` script is *not* supported if run without the
necessary pip/setuptools/virtualenv distributions available locally. All
of the installation methods above include a ``virtualenv_support``
directory alongside ``virtualenv.py`` which contains a complete set of
pip and setuptools distributions, and so are fully supported.

331
docs/reference.rst Normal file
View file

@ -0,0 +1,331 @@
Reference Guide
===============
``virtualenv`` Command
----------------------
.. _usage:
Usage
~~~~~
:command:`virtualenv [OPTIONS] ENV_DIR`
Where ``ENV_DIR`` is an absolute or relative path to a directory to create
the virtual environment in.
.. _options:
Options
~~~~~~~
.. program: virtualenv
.. option:: --version
show program's version number and exit
.. option:: -h, --help
show this help message and exit
.. option:: -v, --verbose
Increase verbosity.
.. option:: -q, --quiet
Decrease verbosity.
.. option:: -p PYTHON_EXE, --python=PYTHON_EXE
The Python interpreter to use, e.g.,
``--python=python2.5`` will use the python2.5 interpreter
to create the new environment. The default is the
interpreter that virtualenv was installed with
(like ``/usr/bin/python``)
.. option:: --clear
Clear out the non-root install and start from scratch.
.. option:: --system-site-packages
Give the virtual environment access to the global
site-packages.
.. option:: --always-copy
Always copy files rather than symlinking.
.. option:: --relocatable
Make an EXISTING virtualenv environment relocatable.
This fixes up scripts and makes all .pth files relative.
.. option:: --unzip-setuptools
Unzip Setuptools when installing it.
.. option:: --no-setuptools
Do not install setuptools in the new virtualenv.
.. option:: --no-pip
Do not install pip in the new virtualenv.
.. option:: --no-wheel
Do not install wheel in the new virtualenv.
.. option:: --extra-search-dir=DIR
Directory to look for setuptools/pip distributions in.
This option can be specified multiple times.
.. option:: --prompt=PROMPT
Provides an alternative prompt prefix for this
environment.
.. option:: --download
Download preinstalled packages from PyPI.
.. option:: --no-download
Do not download preinstalled packages from PyPI.
.. option:: --no-site-packages
DEPRECATED. Retained only for backward compatibility.
Not having access to global site-packages is now the
default behavior.
.. option:: --distribute
.. option:: --setuptools
Legacy; now have no effect. Before version 1.10 these could be used
to choose whether to install Distribute_ or Setuptools_ into the created
virtualenv. Distribute has now been merged into Setuptools, and the
latter is always installed.
.. _Distribute: https://pypi.org/project/distribute
.. _Setuptools: https://pypi.org/project/setuptools
Configuration
-------------
Environment Variables
~~~~~~~~~~~~~~~~~~~~~
Each command line option is automatically used to look for environment
variables with the name format ``VIRTUALENV_<UPPER_NAME>``. That means
the name of the command line options are capitalized and have dashes
(``'-'``) replaced with underscores (``'_'``).
For example, to automatically use a custom Python binary instead of the
one virtualenv is run with you can also set an environment variable::
$ export VIRTUALENV_PYTHON=/opt/python-3.3/bin/python
$ virtualenv ENV
It's the same as passing the option to virtualenv directly::
$ virtualenv --python=/opt/python-3.3/bin/python ENV
This also works for appending command line options, like ``--find-links``.
Just leave an empty space between the passed values, e.g.::
$ export VIRTUALENV_EXTRA_SEARCH_DIR="/path/to/dists /path/to/other/dists"
$ virtualenv ENV
is the same as calling::
$ virtualenv --extra-search-dir=/path/to/dists --extra-search-dir=/path/to/other/dists ENV
.. envvar:: VIRTUAL_ENV_DISABLE_PROMPT
Any virtualenv *activated* when this is set to a non-empty value will leave
the shell prompt unchanged during processing of the
:ref:`activate script <activate>`, rather than modifying it to indicate
the newly activated environment.
Configuration File
~~~~~~~~~~~~~~~~~~
virtualenv also looks for a standard ini config file. On Unix and Mac OS X
that's ``$HOME/.virtualenv/virtualenv.ini`` and on Windows, it's
``%APPDATA%\virtualenv\virtualenv.ini``.
The names of the settings are derived from the long command line option,
e.g. the option :option:`--python <-p>` would look like this::
[virtualenv]
python = /opt/python-3.3/bin/python
Appending options like :option:`--extra-search-dir` can be written on multiple
lines::
[virtualenv]
extra-search-dir =
/path/to/dists
/path/to/other/dists
Please have a look at the output of :option:`--help <-h>` for a full list
of supported options.
Extending Virtualenv
--------------------
Creating Your Own Bootstrap Scripts
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
While this creates an environment, it doesn't put anything into the
environment. Developers may find it useful to distribute a script
that sets up a particular environment, for example a script that
installs a particular web application.
.. note::
A bootstrap script requires a ``virtualenv_support`` directory containing
``pip`` and ``setuptools`` wheels alongside it, just like the actual virtualenv
script. Running a bootstrap script without a ``virtualenv_support`` directory
is unsupported (but if you use ``--no-setuptools`` and manually install ``pip``
and ``setuptools`` in your virtualenv, it will work).
To create a script like this, call
:py:func:`virtualenv.create_bootstrap_script`, and write the
result to your new bootstrapping script.
.. py:function:: create_bootstrap_script(extra_text)
Creates a bootstrap script from ``extra_text``, which is like
this script but with extend_parser, adjust_options, and after_install hooks.
This returns a string that (written to disk of course) can be used
as a bootstrap script with your own customizations. The script
will be the standard virtualenv.py script, with your extra text
added (your extra text should be Python code).
If you include these functions, they will be called:
.. py:function:: extend_parser(optparse_parser)
You can add or remove options from the parser here.
.. py:function:: adjust_options(options, args)
You can change options here, or change the args (if you accept
different kinds of arguments, be sure you modify ``args`` so it is
only ``[DEST_DIR]``).
.. py:function:: after_install(options, home_dir)
After everything is installed, this function is called. This
is probably the function you are most likely to use. An
example would be::
def after_install(options, home_dir):
if sys.platform == 'win32':
bin = 'Scripts'
else:
bin = 'bin'
subprocess.call([join(home_dir, bin, 'easy_install'),
'MyPackage'])
subprocess.call([join(home_dir, bin, 'my-package-script'),
'setup', home_dir])
This example immediately installs a package, and runs a setup
script from that package.
Bootstrap Example
~~~~~~~~~~~~~~~~~
Here's a more concrete example of how you could use this::
import virtualenv, textwrap
output = virtualenv.create_bootstrap_script(textwrap.dedent("""
import os, subprocess
def after_install(options, home_dir):
etc = join(home_dir, 'etc')
if not os.path.exists(etc):
os.makedirs(etc)
subprocess.call([join(home_dir, 'bin', 'easy_install'),
'BlogApplication'])
subprocess.call([join(home_dir, 'bin', 'paster'),
'make-config', 'BlogApplication',
join(etc, 'blog.ini')])
subprocess.call([join(home_dir, 'bin', 'paster'),
'setup-app', join(etc, 'blog.ini')])
"""))
f = open('blog-bootstrap.py', 'w').write(output)
Another example is available `here`__.
.. __: https://github.com/socialplanning/fassembler/blob/master/fassembler/create-venv-script.py
Compatibility with the stdlib venv module
-----------------------------------------
Starting with Python 3.3, the Python standard library includes a ``venv``
module that provides similar functionality to ``virtualenv`` - however, the
mechanisms used by the two modules are very different.
Problems arise when environments get "nested" (a virtual environment is
created from within another one - for example, running the virtualenv tests
using tox, where tox creates a virtual environment to run the tests, and the
tests themselves create further virtual environments).
``virtualenv`` supports creating virtual environments from within another one
(the ``sys.real_prefix`` variable allows ``virtualenv`` to locate the "base"
environment) but stdlib-style ``venv`` environments don't use that mechanism,
so explicit support is needed for those environments.
A standard library virtual environment is most easily identified by checking
``sys.prefix`` and ``sys.base_prefix``. If these differ, the interpreter is
running in a virtual environment and the base interpreter is located in the
directory specified by ``sys.base_prefix``. Therefore, when
``sys.base_prefix`` is set, virtualenv gets the interpreter files from there
rather than from ``sys.prefix`` (in the same way as ``sys.real_prefix`` is
used for virtualenv-style environments). In practice, this is sufficient for
all platforms other than Windows.
On Windows from Python 3.7.2 onwards, a stdlib-style virtual environment does
not contain an actual Python interpreter executable, but rather a "redirector"
which launches the actual interpreter from the base environment (this
redirector is based on the same code as the standard ``py.exe`` launcher). As
a result, the virtualenv approach of copying the interpreter from the starting
environment fails. In order to correctly set up the virtualenv, therefore, we
need to be running from a "full" environment. To ensure that, we re-invoke the
``virtualenv.py`` script using the "base" interpreter, in the same way as we
do with the ``--python`` command line option.
The process of identifying the base interpreter is complicated by the fact
that the implementation changed between different Python versions. The
logic used is as follows:
1. If the (private) attribute ``sys._base_executable`` is present, this is
the base interpreter. This is the long-term solution and should be stable
in the future (the attribute may become public, and have the leading
underscore removed, in a Python 3.8, but that is not confirmed yet).
2. In the absence of ``sys._base_executable`` (only the case for Python 3.7.2)
we check for the existence of the environment variable
``__PYVENV_LAUNCHER__``. This is used by the redirector, and if it is
present, we know that we are in a stdlib-style virtual environment and need
to locate the base Python. In most cases, the base environment is located
at ``sys.base_prefix`` - however, in the case where the user creates a
virtualenv, and then creates a venv from that virtualenv,
``sys.base_prefix`` is not correct - in that case, though, we have
``sys.real_prefix`` (set by virtualenv) which *is* correct.
There is one further complication - as noted above, the environment variable
``__PYVENV_LAUNCHER__`` affects how the interpreter works, so before we
re-invoke the virtualenv script, we remove this from the environment.

280
docs/userguide.rst Normal file
View file

@ -0,0 +1,280 @@
User Guide
==========
Usage
-----
Virtualenv has one basic command::
$ virtualenv ENV
Where ``ENV`` is a directory in which to place the new virtual environment. It has
a number of usual effects (modifiable by many :ref:`options`):
- :file:`ENV/lib/` and :file:`ENV/include/` are created, containing supporting
library files for a new virtualenv python. Packages installed in this
environment will live under :file:`ENV/lib/pythonX.X/site-packages/`.
- :file:`ENV/bin` is created, where executables live - noticeably a new
:command:`python`. Thus running a script with ``#! /path/to/ENV/bin/python``
would run that script under this virtualenv's python.
- The crucial packages pip_ and setuptools_ are installed, which allow other
packages to be easily installed to the environment. This associated pip
can be run from :file:`ENV/bin/pip`.
The python in your new virtualenv is effectively isolated from the python that
was used to create it.
.. _pip: https://pypi.org/project/pip
.. _setuptools: https://pypi.org/project/setuptools
.. _activate:
activate script
~~~~~~~~~~~~~~~
In a newly created virtualenv there will also be a :command:`activate` shell
script. For Windows systems, activation scripts are provided for
the Command Prompt and Powershell.
On Posix systems, this resides in :file:`ENV/bin/`, so you can run::
$ source /path/to/ENV/bin/activate
For some shells (e.g. the original Bourne Shell) you may need to use the
:command:`.` command, when :command:`source` does not exist. There are also
separate activate files for some other shells, like csh and fish.
:file:`bin/activate` should work for bash/zsh/dash.
This will change your ``$PATH`` so its first entry is the virtualenv's
``bin/`` directory. (You have to use ``source`` because it changes your
shell environment in-place.) This is all it does; it's purely a
convenience.
If you directly run a script or the python interpreter
from the virtualenv's ``bin/`` directory (e.g. ``path/to/ENV/bin/pip``
or ``/path/to/ENV/bin/python-script.py``) then ``sys.path`` will
automatically be set to use the Python libraries associated with the
virtualenv. But, unlike the activation scripts, the environment variables
``PATH`` and ``VIRTUAL_ENV`` will *not* be modified. This means that if
your Python script uses e.g. ``subprocess`` to run another Python script
(e.g. via a ``#!/usr/bin/env python`` shebang line) the second script
*may not be executed with the same Python binary as the first* nor have
the same libraries available to it. To avoid this happening your first
script will need to modify the environment variables in the same manner
as the activation scripts, before the second script is executed.
The ``activate`` script will also modify your shell prompt to indicate
which environment is currently active. To disable this behaviour, see
:envvar:`VIRTUAL_ENV_DISABLE_PROMPT`.
To undo these changes to your path (and prompt), just run::
$ deactivate
On Windows, the equivalent ``activate`` script is in the ``Scripts`` folder::
> \path\to\env\Scripts\activate
And type ``deactivate`` to undo the changes.
Based on your active shell (CMD.exe or Powershell.exe), Windows will use
either activate.bat or activate.ps1 (as appropriate) to activate the
virtual environment. If using Powershell, see the notes about code signing
below.
.. note::
If using Powershell, the ``activate`` script is subject to the
`execution policies`_ on the system. By default on Windows 7, the system's
execution policy is set to ``Restricted``, meaning no scripts like the
``activate`` script are allowed to be executed. But that can't stop us
from changing that slightly to allow it to be executed.
In order to use the script, you can relax your system's execution
policy to ``AllSigned``, meaning all scripts on the system must be
digitally signed to be executed. Since the virtualenv activation
script is signed by one of the authors (Jannis Leidel) this level of
the execution policy suffices. As an administrator run::
PS C:\> Set-ExecutionPolicy AllSigned
Then you'll be asked to trust the signer, when executing the script.
You will be prompted with the following::
PS C:\> virtualenv .\foo
New python executable in C:\foo\Scripts\python.exe
Installing setuptools................done.
Installing pip...................done.
PS C:\> .\foo\scripts\activate
Do you want to run software from this untrusted publisher?
File C:\foo\scripts\activate.ps1 is published by E=jannis@leidel.info,
CN=Jannis Leidel, L=Berlin, S=Berlin, C=DE, Description=581796-Gh7xfJxkxQSIO4E0
and is not trusted on your system. Only run scripts from trusted publishers.
[V] Never run [D] Do not run [R] Run once [A] Always run [?] Help
(default is "D"):A
(foo) PS C:\>
If you select ``[A] Always Run``, the certificate will be added to the
Trusted Publishers of your user account, and will be trusted in this
user's context henceforth. If you select ``[R] Run Once``, the script will
be run, but you will be prompted on a subsequent invocation. Advanced users
can add the signer's certificate to the Trusted Publishers of the Computer
account to apply to all users (though this technique is out of scope of this
document).
Alternatively, you may relax the system execution policy to allow running
of local scripts without verifying the code signature using the following::
PS C:\> Set-ExecutionPolicy RemoteSigned
Since the ``activate.ps1`` script is generated locally for each virtualenv,
it is not considered a remote script and can then be executed.
On xonsh, the equivalent ``activate`` script is called ``activate.xsh``, and
lives in either the ``bin/`` directory (on posix systems) or the ``Scripts\``
directory (on Windows). For example::
$ source /path/to/ENV/bin/activate.xsh
With xonsh, you may still run the ``deactivate`` command to undo the changes.
.. _`execution policies`: http://technet.microsoft.com/en-us/library/dd347641.aspx
Removing an Environment
~~~~~~~~~~~~~~~~~~~~~~~
Removing a virtual environment is simply done by deactivating it and deleting the
environment folder with all its contents::
(ENV)$ deactivate
$ rm -r /path/to/ENV
The :option:`--system-site-packages` Option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you build with ``virtualenv --system-site-packages ENV``, your virtual
environment will inherit packages from ``/usr/lib/python2.7/site-packages``
(or wherever your global site-packages directory is).
This can be used if you have control over the global site-packages directory,
and you want to depend on the packages there. If you want isolation from the
global system, do not use this flag.
If you need to change this option after creating a virtual environment, you can
add (to turn off) or remove (to turn on) the file ``no-global-site-packages.txt``
from ``lib/python3.7/`` or equivalent in the environments directory.
Windows Notes
~~~~~~~~~~~~~
Some paths within the virtualenv are slightly different on Windows: scripts and
executables on Windows go in ``ENV\Scripts\`` instead of ``ENV/bin/`` and
libraries go in ``ENV\Lib\`` rather than ``ENV/lib/``.
To create a virtualenv under a path with spaces in it on Windows, you'll need
the `win32api <https://github.com/mhammond/pywin32/>`_ library installed.
Using Virtualenv without ``bin/python``
---------------------------------------
Sometimes you can't or don't want to use the Python interpreter
created by the virtualenv. For instance, in a `mod_python
<http://www.modpython.org/>`_ or `mod_wsgi <http://www.modwsgi.org/>`_
environment, there is only one interpreter.
Luckily, it's easy. You must use the custom Python interpreter to
*install* libraries. But to *use* libraries, you just have to be sure
the path is correct. A script is available to correct the path. You
can setup the environment like::
activate_this = '/path/to/env/bin/activate_this.py'
exec(open(activate_this).read(), {'__file__': activate_this})
This will change ``sys.path`` and even change ``sys.prefix``, but also allow
you to use an existing interpreter. Items in your environment will show up
first on ``sys.path``, before global items. However, global items will
always be accessible (as if the :option:`--system-site-packages` flag had been
used in creating the environment, whether it was or not). Also, this cannot undo
the activation of other environments, or modules that have been imported.
You shouldn't try to, for instance, activate an environment before a web
request; you should activate *one* environment as early as possible, and not
do it again in that process.
Making Environments Relocatable
-------------------------------
**Note:** this option is somewhat experimental, and there are probably
caveats that have not yet been identified.
.. warning::
The ``--relocatable`` option currently has a number of issues,
and is not guaranteed to work in all circumstances. It is possible
that the option will be deprecated in a future version of ``virtualenv``.
Normally environments are tied to a specific path. That means that
you cannot move an environment around or copy it to another computer.
You can fix up an environment to make it relocatable with the
command::
$ virtualenv --relocatable ENV
This will make some of the files created by setuptools use relative paths,
and will change all the scripts to use ``activate_this.py`` instead of using
the location of the Python interpreter to select the environment.
**Note:** scripts which have been made relocatable will only work if
the virtualenv is activated, specifically the python executable from
the virtualenv must be the first one on the system PATH. Also note that
the activate scripts are not currently made relocatable by
``virtualenv --relocatable``.
**Note:** you must run this after you've installed *any* packages into
the environment. If you make an environment relocatable, then
install a new package, you must run ``virtualenv --relocatable``
again.
Also, this **does not make your packages cross-platform**. You can
move the directory around, but it can only be used on other similar
computers. Some known environmental differences that can cause
incompatibilities: a different version of Python, when one platform
uses UCS2 for its internal unicode representation and another uses
UCS4 (a compile-time option), obvious platform changes like Windows
vs. Linux, or Intel vs. ARM, and if you have libraries that bind to C
libraries on the system, if those C libraries are located somewhere
different (either different versions, or a different filesystem
layout).
If you use this flag to create an environment, currently, the
:option:`--system-site-packages` option will be implied.
The :option:`--extra-search-dir` option
---------------------------------------
This option allows you to provide your own versions of setuptools and/or
pip to use instead of the embedded versions that come with virtualenv.
To use this feature, pass one or more ``--extra-search-dir`` options to
virtualenv like this::
$ virtualenv --extra-search-dir=/path/to/distributions ENV
The ``/path/to/distributions`` path should point to a directory that contains
setuptools and/or pip wheels.
virtualenv will look for wheels in the specified directories, but will use
pip's standard algorithm for selecting the wheel to install, which looks for
the latest compatible wheel.
As well as the extra directories, the search order includes:
#. The ``virtualenv_support`` directory relative to virtualenv.py
#. The directory where virtualenv.py is located.
#. The current directory.

52
pyproject.toml Normal file
View file

@ -0,0 +1,52 @@
[build-system]
requires = [
"setuptools >= 40.0.0",
"wheel >= 0.29.0",
"setuptools-scm >= 2, < 4",
]
build-backend = 'setuptools.build_meta'
[tool.black]
line-length = 120
[tool.towncrier]
package = "virtualenv"
filename = "docs/changes.rst"
directory = "docs/changelog"
template = "docs/changelog/template.jinja2"
title_format = "v{version} ({project_date})"
issue_format = "`#{issue} <https://github.com/pypa/virtualenv/issues/{issue}>`_"
underlines = ["-", "^"]
[[tool.towncrier.section]]
path = ""
[[tool.towncrier.type]]
directory = "bugfix"
name = "Bugfixes"
showcontent = true
[[tool.towncrier.type]]
directory = "feature"
name = "Features"
showcontent = true
[[tool.towncrier.type]]
directory = "deprecation"
name = "Deprecations (removal in next major release)"
showcontent = true
[[tool.towncrier.type]]
directory = "breaking"
name = "Backward incompatible changes"
showcontent = true
[[tool.towncrier.type]]
directory = "doc"
name = "Documentation"
showcontent = true
[[tool.towncrier.type]]
directory = "misc"
name = "Miscellaneous"
showcontent = true

8
readthedocs.yml Normal file
View file

@ -0,0 +1,8 @@
build:
image: latest
python:
version: 3.6
pip_install: true
extra_requirements:
- docs
formats: []

94
setup.cfg Normal file
View file

@ -0,0 +1,94 @@
[metadata]
name = virtualenv
version = attr: virtualenv.__version__
description = Virtual Python Environment builder
long_description = file: README.rst
keywords = virtual, environments, isolated
maintainer = Bernat Gabor
author = Bernat Gabor
maintainer-email = gaborjbernat@gmail.com
author-email = gaborjbernat@gmail.com
url = https://virtualenv.pypa.io/
project_urls =
Source=https://github.com/pypa/virtualenv
Tracker=https://github.com/pypa/virtualenv/issues
classifiers =
Development Status :: 3 - Alpha
Programming Language :: Python :: 3.8
Programming Language :: Python :: 3.7
Programming Language :: Python :: 3.6
Programming Language :: Python :: 3.5
Programming Language :: Python :: 3.4
Programming Language :: Python :: 3
Programming Language :: Python :: 2
Programming Language :: Python :: 2.7
Intended Audience :: Developers
License :: OSI Approved :: MIT License
Operating System :: POSIX
Operating System :: Microsoft :: Windows
Operating System :: MacOS :: MacOS X
Topic :: Software Development :: Testing
Topic :: Software Development :: Libraries
Topic :: Utilities
platforms = any
license = MIT
license_file = LICENSE.txt
[options]
packages = find:
package_dir =
=src
zip_safe = True
python_requires = >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*
install_requires =
six >= 1.12.0, < 2
pathlib2 >= 2.3.3, < 3
appdirs >= 1.4.3
entrypoints >= 0.3, <1
distlib >= 0.3.0, <1; sys.platform == 'win32'
[options.packages.find]
where = src
[options.extras_require]
testing =
pytest >= 4.0.0, <6
coverage >= 4.5.0, < 5
pytest-mock
docs =
sphinx >= 2.0.0, < 3
towncrier >= 18.5.0
sphinx_rtd_theme >= 0.4.2, < 1
[options.package_data]
virtualenv.seed.embed.wheels = *.whl
[options.entry_points]
console_scripts =
virtualenv=virtualenv.__main__:run
virtualenv.discovery =
builtin = virtualenv.interpreters.discovery.builtin:Builtin
virtualenv.create =
cpython3-posix = virtualenv.interpreters.create.cpython.cpython3:CPython3Posix
cpython3-win = virtualenv.interpreters.create.cpython.cpython3:CPython3Windows
cpython2-posix = virtualenv.interpreters.create.cpython.cpython2:CPython2Posix
cpython2-win = virtualenv.interpreters.create.cpython.cpython2:CPython2Windows
venv = virtualenv.interpreters.create.venv:Venv
virtualenv.seed =
none = virtualenv.seed.none:NoneSeeder
pip = virtualenv.seed.embed.pip_invoke:PipInvoke
link-app-data = virtualenv.seed.embed.link_app_data:LinkFromAppData
virtualenv.activate =
[sdist]
formats = gztar
[bdist_wheel]
universal = true
[tool:pytest]
markers =
bash
csh
fish
pwsh
xonsh
junit_family = xunit2

17
setup.py Normal file
View file

@ -0,0 +1,17 @@
# -*- coding: utf-8 -*-
import textwrap
from setuptools import setup
setup(
use_scm_version={
"write_to": "src/virtualenv/version.py",
"write_to_template": textwrap.dedent(
"""
# coding: utf-8
from __future__ import unicode_literals
__version__ = {version!r}
"""
).lstrip(),
}
)

5
src/virtualenv/__init__.py Normal file
View file

@ -0,0 +1,5 @@
from __future__ import absolute_import, unicode_literals
from .version import __version__
__all__ = ("__version__", "run")

22
src/virtualenv/__main__.py Normal file
View file

@ -0,0 +1,22 @@
from __future__ import absolute_import, print_function, unicode_literals
import sys
from virtualenv.error import ProcessCallFailed
from virtualenv.run import run_via_cli
def run(args=None):
if args is None:
args = sys.argv[1:]
try:
run_via_cli(args)
except ProcessCallFailed as exception:
print("subprocess call failed for {}".format(exception.cmd))
print(exception.out, file=sys.stdout, end="")
print(exception.err, file=sys.stderr, end="")
raise SystemExit(exception.code)
if __name__ == "__main__":
run()

1
src/virtualenv/activation/__init__.py Normal file
View file

@ -0,0 +1 @@
from __future__ import absolute_import, unicode_literals

24
src/virtualenv/activation/activator.py Normal file
View file

@ -0,0 +1,24 @@
from __future__ import absolute_import, unicode_literals
from abc import ABCMeta, abstractmethod
import six
@six.add_metaclass(ABCMeta)
class Activator(object):
def __init__(self, options):
self.flag_prompt = options.prompt
@classmethod
def add_parser_arguments(cls, parser):
pass
def run(self, creator):
self.generate()
if self.flag_prompt is not None:
creator.pyenv_cfg["prompt"] = self.flag_prompt
@abstractmethod
def generate(self):
raise NotImplementedError

1
src/virtualenv/config/__init__.py Normal file
View file

@ -0,0 +1 @@
from __future__ import absolute_import, unicode_literals

1
src/virtualenv/config/cli/__init__.py Normal file
View file

@ -0,0 +1 @@
from __future__ import absolute_import, unicode_literals

64
src/virtualenv/config/cli/parser.py Normal file
View file

@ -0,0 +1,64 @@
from __future__ import absolute_import, unicode_literals
from argparse import SUPPRESS, ArgumentDefaultsHelpFormatter, ArgumentParser
from ..env_var import get_env_var
from ..ini import IniConfig
class VirtualEnvConfigParser(ArgumentParser):
"""
Custom option parser which updates its defaults by checking the configuration files and environmental variables
"""
def __init__(self, *args, **kwargs):
self.file_config = IniConfig()
self.epilog_list = []
kwargs["epilog"] = self.file_config.epilog
kwargs["add_help"] = False
kwargs["formatter_class"] = HelpFormatter
kwargs["prog"] = "virtualenv"
super(VirtualEnvConfigParser, self).__init__(*args, **kwargs)
self._fixed = set()
def _fix_defaults(self):
for action in self._actions:
action_id = id(action)
if action_id not in self._fixed:
self._fix_default(action)
self._fixed.add(action_id)
def _fix_default(self, action):
if hasattr(action, "default") and hasattr(action, "dest") and action.default != SUPPRESS:
as_type = type(action.default)
outcome = get_env_var(action.dest, as_type)
if outcome is None and self.file_config:
outcome = self.file_config.get(action.dest, as_type)
if outcome is not None:
action.default, action.default_source = outcome
def enable_help(self):
self._fix_defaults()
self.add_argument("-h", "--help", action="help", default=SUPPRESS, help="show this help message and exit")
def parse_known_args(self, args=None, namespace=None):
self._fix_defaults()
return super(VirtualEnvConfigParser, self).parse_known_args(args, namespace=namespace)
def parse_args(self, args=None, namespace=None):
self._fix_defaults()
return super(VirtualEnvConfigParser, self).parse_args(args, namespace=namespace)
class HelpFormatter(ArgumentDefaultsHelpFormatter):
def __init__(self, prog):
super(HelpFormatter, self).__init__(prog, max_help_position=35, width=240)
def _get_help_string(self, action):
# noinspection PyProtectedMember
text = super(HelpFormatter, self)._get_help_string(action)
if hasattr(action, "default_source"):
default = " (default: %(default)s)"
if text.endswith(default):
text = "{} (default: %(default)s -> from %(default_source)s)".format(text[: -len(default)])
return text

69
src/virtualenv/config/convert.py Normal file
View file

@ -0,0 +1,69 @@
from __future__ import absolute_import, unicode_literals
import logging
BOOLEAN_STATES = {
"1": True,
"yes": True,
"true": True,
"on": True,
"0": False,
"no": False,
"false": False,
"off": False,
}
def _convert_to_boolean(value):
if value.lower() not in BOOLEAN_STATES:
raise ValueError("Not a boolean: %s" % value)
return BOOLEAN_STATES[value.lower()]
def _expand_to_list(value):
if isinstance(value, (str, bytes)):
value = filter(None, [x.strip() for x in value.splitlines()])
return list(value)
def _as_list(value, flatten=True):
values = _expand_to_list(value)
if not flatten:
return values # pragma: no cover
result = []
for value in values:
sub_values = value.split()
result.extend(sub_values)
return result
def _as_none(value):
if not value:
return None
return str(value)
CONVERT = {bool: _convert_to_boolean, list: _as_list, type(None): _as_none}
def _get_converter(as_type):
for of_type, func in CONVERT.items():
if issubclass(as_type, of_type):
getter = func
break
else:
getter = as_type
return getter
def convert(value, as_type, source):
"""Convert the value as a given type where the value comes from the given source"""
getter = _get_converter(as_type)
try:
return getter(value)
except Exception as exception:
logging.warning("%s failed to convert %r as %r because %r", source, value, getter, exception)
raise
__all__ = ("convert",)

27
src/virtualenv/config/env_var.py Normal file
View file

@ -0,0 +1,27 @@
from __future__ import absolute_import, unicode_literals
import os
from .convert import convert
def get_env_var(key, as_type):
"""Get the environment variable option.
:param key: the config key requested
:param as_type: the type we would like to convert it to
:return:
"""
environ_key = "VIRTUALENV_{}".format(key.upper())
if environ_key in os.environ:
value = os.environ[environ_key]
# noinspection PyBroadException
try:
source = "env var {}".format(environ_key)
as_type = convert(value, as_type, source)
return as_type, source
except Exception:
pass
__all__ = ("get_env_var",)

75
src/virtualenv/config/ini.py Normal file
View file

@ -0,0 +1,75 @@
from __future__ import absolute_import, unicode_literals
import logging
import os
from pathlib2 import Path
from virtualenv.info import PY3, get_default_config_dir
from .convert import convert
try:
import ConfigParser
except ImportError:
# noinspection PyPep8Naming
import configparser as ConfigParser
DEFAULT_CONFIG_FILE = get_default_config_dir() / "virtualenv.ini"
class IniConfig(object):
VIRTUALENV_CONFIG_FILE_ENV_VAR = str("VIRTUALENV_CONFIG_FILE")
STATE = {None: "failed to parse", True: "active", False: "missing"}
section = "virtualenv"
def __init__(self):
config_file = os.environ.get(self.VIRTUALENV_CONFIG_FILE_ENV_VAR, None)
self.is_env_var = config_file is not None
self.config_file = Path(config_file) if config_file is not None else DEFAULT_CONFIG_FILE
self._cache = {}
self.has_config_file = self.config_file.exists()
if self.has_config_file:
self.config_file = self.config_file.resolve()
self.config_parser = ConfigParser.ConfigParser()
try:
with self.config_file.open("rt") as file_handler:
reader = getattr(self.config_parser, "read_file" if PY3 else "readfp")
reader(file_handler)
self.has_virtualenv_section = self.config_parser.has_section(self.section)
except Exception as exception:
logging.error("failed to read config file %s because %r", config_file, exception)
self.has_config_file = None
def get(self, key, as_type):
cache_key = key, as_type
if cache_key in self._cache:
result = self._cache[cache_key]
else:
# noinspection PyBroadException
try:
source = "file"
raw_value = self.config_parser.get(self.section, key.lower())
value = convert(raw_value, as_type, source)
result = value, source
except Exception:
result = None
self._cache[cache_key] = result
return result
def __bool__(self):
return bool(self.has_config_file) and bool(self.has_virtualenv_section)
@property
def epilog(self):
msg = "{}config file {} {} (change{} via env var {})"
return msg.format(
os.linesep,
self.config_file,
self.STATE[self.has_config_file],
"d" if self.is_env_var else "",
self.VIRTUALENV_CONFIG_FILE_ENV_VAR,
)

12
src/virtualenv/error.py Normal file
View file

@ -0,0 +1,12 @@
"""Errors"""
from __future__ import absolute_import, unicode_literals
class ProcessCallFailed(RuntimeError):
"""Failed a process call"""
def __init__(self, code, out, err, cmd):
self.code = code
self.out = out
self.err = err
self.cmd = cmd

25
src/virtualenv/info.py Normal file
View file

@ -0,0 +1,25 @@
from __future__ import absolute_import, unicode_literals
import sys
from appdirs import user_config_dir, user_data_dir
from pathlib2 import Path
IS_PYPY = hasattr(sys, "pypy_version_info")
PY3 = sys.version_info[0] == 3
IS_WIN = sys.platform == "win32"
_DATA_DIR = Path(user_data_dir(appname="virtualenv", appauthor="pypa"))
_CONFIG_DIR = Path(user_config_dir(appname="virtualenv", appauthor="pypa"))
def get_default_data_dir():
return _DATA_DIR
def get_default_config_dir():
return _CONFIG_DIR
__all__ = ("IS_PYPY", "PY3", "IS_WIN", "get_default_data_dir", "get_default_config_dir")

1
src/virtualenv/interpreters/__init__.py Normal file
View file

@ -0,0 +1 @@
from __future__ import absolute_import, unicode_literals

1
src/virtualenv/interpreters/create/__init__.py Normal file
View file

@ -0,0 +1 @@
from __future__ import absolute_import, unicode_literals

1
src/virtualenv/interpreters/create/cpython/__init__.py Normal file
View file

@ -0,0 +1 @@
from __future__ import absolute_import, unicode_literals

137
src/virtualenv/interpreters/create/cpython/common.py Normal file
View file

@ -0,0 +1,137 @@
from __future__ import absolute_import, unicode_literals
import abc
from os import X_OK, access, chmod
import six
from pathlib2 import Path
from virtualenv.interpreters.create.via_global_ref import ViaGlobalRef
from virtualenv.util import copy, ensure_dir, symlink
@six.add_metaclass(abc.ABCMeta)
class CPython(ViaGlobalRef):
def __init__(self, options, interpreter):
super(CPython, self).__init__(options, interpreter)
self.copier = symlink if self.symlinks is True else copy
@classmethod
def supports(cls, interpreter):
return interpreter.implementation == "CPython"
def create(self):
for directory in self.ensure_directories():
ensure_dir(directory)
self.set_pyenv_cfg()
self.pyenv_cfg.write()
true_system_site = self.system_site_package
try:
self.system_site_package = False
self.setup_python()
finally:
if true_system_site != self.system_site_package:
self.system_site_package = true_system_site
def ensure_directories(self):
dirs = [self.env_dir, self.bin_dir]
dirs.extend(self.site_packages)
return dirs
def setup_python(self):
python_dir = Path(self.interpreter.system_executable).parent
for name in self.exe_names():
self.add_executable(python_dir, self.bin_dir, name)
@abc.abstractmethod
def lib_name(self):
raise NotImplementedError
@property
def lib_base(self):
raise NotImplementedError
@property
def lib_dir(self):
return self.env_dir / self.lib_base
@property
def system_stdlib(self):
return Path(self.interpreter.system_prefix) / self.lib_base
def exe_names(self):
yield Path(self.interpreter.system_executable).name
def add_exe_method(self):
if self.copier is symlink:
return self.symlink_exe
return self.copier
@staticmethod
def symlink_exe(src, dest):
symlink(src, dest)
dest_str = str(dest)
if not access(dest_str, X_OK):
chmod(dest_str, 0o755) # pragma: no cover
def add_executable(self, src, dest, name):
src_ex = src / name
if src_ex.exists():
add_exe_method_ = self.add_exe_method()
add_exe_method_(src_ex, dest / name)
@six.add_metaclass(abc.ABCMeta)
class CPythonPosix(CPython):
"""Create a CPython virtual environment on POSIX platforms"""
@classmethod
def supports(cls, interpreter):
return super(CPythonPosix, cls).supports(interpreter) and interpreter.os == "posix"
@property
def bin_name(self):
return "bin"
@property
def lib_name(self):
return "lib"
@property
def lib_base(self):
return Path(self.lib_name) / self.interpreter.python_name
def setup_python(self):
"""Just create an exe in the provisioned virtual environment skeleton directory"""
super(CPythonPosix, self).setup_python()
major, minor = self.interpreter.version_info.major, self.interpreter.version_info.minor
target = self.bin_dir / next(self.exe_names())
for suffix in ("python", "python{}".format(major), "python{}.{}".format(major, minor)):
path = self.bin_dir / suffix
if not path.exists():
symlink(target, path, relative_symlinks_ok=True)
@six.add_metaclass(abc.ABCMeta)
class CPythonWindows(CPython):
@classmethod
def supports(cls, interpreter):
return super(CPythonWindows, cls).supports(interpreter) and interpreter.os == "nt"
@property
def bin_name(self):
return "Scripts"
@property
def lib_name(self):
return "Lib"
@property
def lib_base(self):
return Path(self.lib_name)
def exe_names(self):
yield Path(self.interpreter.system_executable).name
for name in ["python", "pythonw"]:
for suffix in ["exe"]:
yield "{}.{}".format(name, suffix)

70
src/virtualenv/interpreters/create/cpython/cpython2.py Normal file
View file

@ -0,0 +1,70 @@
from __future__ import absolute_import, unicode_literals
import abc
import six
from pathlib2 import Path
from virtualenv.util import copy
from .common import CPython, CPythonPosix, CPythonWindows
HERE = Path(__file__).absolute().parent
@six.add_metaclass(abc.ABCMeta)
class CPython2(CPython):
"""Create a CPython version 2 virtual environment"""
def set_pyenv_cfg(self):
"""
We directly inject the base prefix and base exec prefix to avoid site.py needing to discover these
from home (which usually is done within the interpreter itself)
"""
super(CPython2, self).set_pyenv_cfg()
self.pyenv_cfg["base-prefix"] = self.interpreter.system_prefix
self.pyenv_cfg["base-exec-prefix"] = self.interpreter.system_exec_prefix
@classmethod
def supports(cls, interpreter):
return super(CPython2, cls).supports(interpreter) and interpreter.version_info.major == 2
def setup_python(self):
super(CPython2, self).setup_python() # install the core first
self.fixup_python2() # now patch
def add_exe_method(self):
return copy
def fixup_python2(self):
"""Perform operations needed to make the created environment work on Python 2"""
# 1. add landmarks for detecting the python home
self.add_module("os")
# 2. install a patched site-package, the default Python 2 site.py is not smart enough to understand pyvenv.cfg,
# so we inject a small shim that can do this
copy(HERE / "site.py", self.lib_dir / "site.py")
def add_module(self, req):
for ext in self.module_extensions:
file_path = "{}.{}".format(req, ext)
self.copier(self.system_stdlib / file_path, self.lib_dir / file_path)
@property
def module_extensions(self):
return ["py", "pyc"]
class CPython2Posix(CPython2, CPythonPosix):
"""CPython 2 on POSIX"""
def fixup_python2(self):
super(CPython2Posix, self).fixup_python2()
# linux needs the lib-dynload, these are builtins on Windows
self.add_folder("lib-dynload")
def add_folder(self, folder):
self.copier(self.system_stdlib / folder, self.lib_dir / folder)
class CPython2Windows(CPython2, CPythonWindows):
"""CPython 2 on Windows"""

37
src/virtualenv/interpreters/create/cpython/cpython3.py Normal file
View file

@ -0,0 +1,37 @@
from __future__ import absolute_import, unicode_literals
import abc
import six
from pathlib2 import Path
from virtualenv.util import copy
from .common import CPython, CPythonPosix, CPythonWindows
@six.add_metaclass(abc.ABCMeta)
class CPython3(CPython):
@classmethod
def supports(cls, interpreter):
return super(CPython3, cls).supports(interpreter) and interpreter.version_info.major == 3
class CPython3Posix(CPythonPosix, CPython3):
""""""
class CPython3Windows(CPythonWindows, CPython3):
""""""
def setup_python(self):
super(CPython3Windows, self).setup_python()
self.include_dll()
def include_dll(self):
dll_folder = Path(self.interpreter.system_prefix) / "DLLs"
host_exe_folder = Path(self.interpreter.system_executable).parent
for folder in [host_exe_folder, dll_folder]:
for file in folder.iterdir():
if file.suffix in (".pyd", ".dll"):
copy(file, self.bin_dir / file.name)

101
src/virtualenv/interpreters/create/cpython/site.py Normal file
View file

@ -0,0 +1,101 @@
"""
A simple shim module to fix up things on Python 2 only.
Note: until we setup correctly the paths we can only import built-ins.
"""
import sys
def main():
"""Patch what needed, and invoke the original site.py"""
config = read_pyvenv()
sys.real_prefix = sys.base_prefix = config["base-prefix"]
sys.base_exec_prefix = config["base-exec-prefix"]
global_site_package_enabled = config.get("include-system-site-packages", False) == "true"
rewrite_standard_library_sys_path()
disable_user_site_package()
load_host_site()
if global_site_package_enabled:
add_global_site_package()
def load_host_site():
"""trigger reload of site.py - now it will use the standard library instance that will take care of init"""
# the standard library will be the first element starting with the real prefix, not zip, must be present
import os
std_lib = os.path.dirname(os.__file__)
std_lib_suffix = std_lib[len(sys.real_prefix) :] # strip away the real prefix to keep just the suffix
reload(sys.modules["site"]) # noqa
# ensure standard library suffix/site-packages is on the new path
# notably Debian derivatives change site-packages constant to dist-packages, so will not get added
target = os.path.join("{}{}".format(sys.prefix, std_lib_suffix), "site-packages")
if target not in reversed(sys.path): # if wasn't automatically added do it explicitly
sys.path.append(target)
def read_pyvenv():
"""read pyvenv.cfg"""
os_sep = "\\" if sys.platform == "win32" else "/" # no os module here yet - poor mans version
config_file = "{}{}pyvenv.cfg".format(sys.prefix, os_sep)
with open(config_file) as file_handler:
lines = file_handler.readlines()
config = {}
for line in lines:
try:
split_at = line.index("=")
except ValueError:
continue # ignore bad/empty lines
else:
config[line[:split_at].strip()] = line[split_at + 1 :].strip()
return config
def rewrite_standard_library_sys_path():
"""Once this site file is loaded the standard library paths have already been set, fix them up"""
sep = "\\" if sys.platform == "win32" else "/"
exe_dir = sys.executable[: sys.executable.rfind(sep)]
for at, value in enumerate(sys.path):
# replace old sys prefix path starts with new
if value == exe_dir:
pass # don't fix the current executable location, notably on Windows this gets added
elif value.startswith(sys.prefix):
value = "{}{}".format(sys.base_prefix, value[len(sys.prefix) :])
elif value.startswith(sys.exec_prefix):
value = "{}{}".format(sys.base_exec_prefix, value[len(sys.exec_prefix) :])
sys.path[at] = value
def disable_user_site_package():
"""Flip the switch on enable user site package"""
# sys.flags is a c-extension type, so we cannot monkey patch it, replace it with a python class to flip it
sys.original_flags = sys.flags
class Flags(object):
def __init__(self):
self.__dict__ = {key: getattr(sys.flags, key) for key in dir(sys.flags) if not key.startswith("_")}
sys.flags = Flags()
sys.flags.no_user_site = 1
def add_global_site_package():
"""add the global site package"""
import site
# add user site package
sys.flags = sys.original_flags # restore original
site.ENABLE_USER_SITE = None # reset user site check
# add the global site package to the path - use new prefix and delegate to site.py
orig_prefixes = None
try:
orig_prefixes = site.PREFIXES
site.PREFIXES = [sys.base_prefix, sys.base_exec_prefix]
site.main()
finally:
site.PREFIXES = orig_prefixes
main()

162
src/virtualenv/interpreters/create/creator.py Normal file
View file

@ -0,0 +1,162 @@
from __future__ import absolute_import, print_function, unicode_literals
import json
import logging
import os
import shutil
from abc import ABCMeta, abstractmethod
from argparse import ArgumentTypeError
from pathlib2 import Path
from six import add_metaclass
from virtualenv.info import IS_WIN
from virtualenv.pyenv_cfg import PyEnvCfg
from virtualenv.util import run_cmd
from virtualenv.version import __version__
HERE = Path(__file__).absolute().parent
DEBUG_SCRIPT = HERE / "debug.py"
@add_metaclass(ABCMeta)
class Creator(object):
def __init__(self, options, interpreter):
self.interpreter = interpreter
self._debug = None
self.dest_dir = Path(options.dest_dir)
self.system_site_package = options.system_site
self.clear = options.clear
self.pyenv_cfg = PyEnvCfg.from_folder(self.dest_dir)
@classmethod
def add_parser_arguments(cls, parser, interpreter):
parser.add_argument(
"--clear",
dest="clear",
action="store_true",
help="clear out the non-root install and start from scratch",
default=False,
)
parser.add_argument(
"--system-site-packages",
default=False,
action="store_true",
dest="system_site",
help="Give the virtual environment access to the system site-packages dir.",
)
def validate_dest_dir(value):
"""No path separator in the path and must be write-able"""
if os.pathsep in value:
raise ArgumentTypeError(
"destination {!r} must not contain the path separator ({}) as this would break "
"the activation scripts".format(value, os.pathsep)
)
value = Path(value)
if value.exists() and value.is_file():
raise ArgumentTypeError("the destination {} already exists and is a file".format(value))
value = dest = value.resolve()
while dest:
if dest.exists():
if os.access(str(dest), os.W_OK):
break
else:
non_write_able(dest, value)
base, _ = dest.parent, dest.name
if base == dest:
non_write_able(dest, value) # pragma: no cover
dest = base
return str(value)
def non_write_able(dest, value):
common = Path(*os.path.commonprefix([value.parts, dest.parts]))
raise ArgumentTypeError(
"the destination {} is not write-able at {}".format(dest.relative_to(common), common)
)
parser.add_argument(
"dest_dir", help="directory to create virtualenv at", type=validate_dest_dir, default="env", nargs="?",
)
def run(self):
if self.dest_dir.exists() and self.clear:
shutil.rmtree(str(self.dest_dir), ignore_errors=True)
self.create()
self.set_pyenv_cfg()
@abstractmethod
def create(self):
raise NotImplementedError
@classmethod
def supports(cls, interpreter):
raise NotImplementedError
def set_pyenv_cfg(self):
self.pyenv_cfg.content = {
"home": self.interpreter.system_exec_prefix,
"include-system-site-packages": "true" if self.system_site_package else "false",
"implementation": self.interpreter.implementation,
"virtualenv": __version__,
}
@property
def env_dir(self):
return Path(self.dest_dir)
@property
def env_name(self):
return self.env_dir.parts[-1]
@property
def bin_name(self):
raise NotImplementedError
@property
def bin_dir(self):
return self.env_dir / self.bin_name
@property
def lib_dir(self):
raise NotImplementedError
@property
def site_packages(self):
return [self.lib_dir / "site-packages"]
@property
def env_exe(self):
return self.bin_dir / "python{}".format(".exe" if IS_WIN else "")
@property
def debug(self):
if self._debug is None:
self._debug = get_env_debug_info(self.env_exe, self.debug_script())
return self._debug
# noinspection PyMethodMayBeStatic
def debug_script(self):
return DEBUG_SCRIPT
def get_env_debug_info(env_exe, debug_script):
cmd = [str(env_exe), str(debug_script)]
logging.debug(" ".join(cmd))
env = os.environ.copy()
env.pop("PYTHONPATH", None)
code, out, err = run_cmd(cmd)
# noinspection PyBroadException
try:
if code != 0:
result = eval(out)
else:
result = json.loads(out)
if err:
result["err"] = err
except Exception as exception:
return {"out": out, "err": err, "returncode": code, "exception": repr(exception)}
if "sys" in result and "path" in result["sys"]:
del result["sys"]["path"][0]
return result

55
src/virtualenv/interpreters/create/debug.py Normal file
View file

@ -0,0 +1,55 @@
"""Inspect a target Python interpreter virtual environment wise"""
import sys # built-in
def run():
"""print debug data about the virtual environment"""
try:
from collections import OrderedDict
except ImportError: # pragma: no cover
# this is possible if the standard library cannot be accessed
# noinspection PyPep8Naming
OrderedDict = dict # pragma: no cover
result = OrderedDict([("sys", OrderedDict())])
for key in (
"executable",
"_base_executable",
"prefix",
"base_prefix",
"real_prefix",
"exec_prefix",
"base_exec_prefix",
"path",
"meta_path",
"version",
):
value = getattr(sys, key, None)
if key == "meta_path" and value is not None:
value = [repr(i) for i in value]
result["sys"][key] = value
import os # landmark
result["os"] = os.__file__
try:
# noinspection PyUnresolvedReferences
import site # site
result["site"] = site.__file__
except ImportError as exception: # pragma: no cover
result["site"] = repr(exception) # pragma: no cover
# try to print out, this will validate if other core modules are available (json in this case)
try:
import json
result["json"] = repr(json)
print(json.dumps(result, indent=2))
except ImportError as exception: # pragma: no cover
result["json"] = repr(exception) # pragma: no cover
print(repr(result)) # pragma: no cover
raise SystemExit(1) # pragma: no cover
if __name__ == "__main__":
run()

0
src/virtualenv/interpreters/create/util.py Normal file
View file

72
src/virtualenv/interpreters/create/venv.py Normal file
View file

@ -0,0 +1,72 @@
from __future__ import absolute_import, unicode_literals
import logging
from copy import copy
from virtualenv.error import ProcessCallFailed
from virtualenv.interpreters.discovery.py_info import CURRENT
from virtualenv.util import run_cmd
from .via_global_ref import ViaGlobalRef
class Venv(ViaGlobalRef):
def __init__(self, options, interpreter):
super(Venv, self).__init__(options, interpreter)
self.can_be_inline = interpreter is CURRENT and interpreter.executable == interpreter.system_executable
self._context = None
@classmethod
def supports(cls, interpreter):
return interpreter.has_venv
def create(self):
if self.can_be_inline:
self.create_inline()
else:
self.create_via_sub_process()
# TODO: cleanup activation scripts
def create_inline(self):
from venv import EnvBuilder
builder = EnvBuilder(
system_site_packages=self.system_site_package,
clear=False,
symlinks=self.symlinks,
with_pip=False,
prompt=None,
)
builder.create(self.dest_dir)
def create_via_sub_process(self):
cmd = self.get_host_create_cmd()
logging.info("create with venv %s", " ".join(cmd))
code, out, err = run_cmd(cmd)
if code != 0:
raise ProcessCallFailed(code, out, err, cmd)
def get_host_create_cmd(self):
cmd = [str(self.interpreter.system_executable), "-m", "venv", "--without-pip"]
if self.system_site_package:
cmd.append("--system-site-packages")
cmd.append("--symlinks" if self.symlinks else "--copies")
cmd.append(str(self.dest_dir))
return cmd
def set_pyenv_cfg(self):
# prefer venv options over ours, but keep our extra
venv_content = copy(self.pyenv_cfg.refresh())
super(Venv, self).set_pyenv_cfg()
self.pyenv_cfg.update(venv_content)
@property
def bin_name(self):
return "Scripts" if self.interpreter.os == "nt" else "bin"
@property
def lib_dir(self):
base = self.dest_dir / ("Lib" if self.interpreter.os == "nt" else "lib")
if self.interpreter.os != "nt":
base = base / self.interpreter.python_name
return base

34
src/virtualenv/interpreters/create/via_global_ref.py Normal file
View file

@ -0,0 +1,34 @@
from __future__ import absolute_import, unicode_literals
from abc import ABCMeta
from six import add_metaclass
from .creator import Creator
@add_metaclass(ABCMeta)
class ViaGlobalRef(Creator):
def __init__(self, options, interpreter):
super(ViaGlobalRef, self).__init__(options, interpreter)
self.symlinks = options.symlinks
@classmethod
def add_parser_arguments(cls, parser, interpreter):
super(ViaGlobalRef, cls).add_parser_arguments(parser, interpreter)
group = parser.add_mutually_exclusive_group()
symlink = False if interpreter.os == "nt" else True
group.add_argument(
"--symlinks",
default=symlink,
action="store_true",
dest="symlinks",
help="Try to use symlinks rather than copies, when symlinks are not the default for the platform.",
)
group.add_argument(
"--copies",
default=not symlink,
action="store_false",
dest="symlinks",
help="Try to use copies rather than symlinks, even when symlinks are the default for the platform.",
)

1
src/virtualenv/interpreters/discovery/__init__.py Normal file
View file

@ -0,0 +1 @@
from __future__ import absolute_import, unicode_literals

76
src/virtualenv/interpreters/discovery/builtin.py Normal file
View file

@ -0,0 +1,76 @@
from __future__ import absolute_import, unicode_literals
import os
import sys
from distutils.spawn import find_executable
from virtualenv.info import IS_WIN
from .discover import Discover
from .py_info import CURRENT, PythonInfo
from .py_spec import PythonSpec
class Builtin(Discover):
def __init__(self, options):
super(Builtin, self).__init__()
self.python_spec = options.python
@classmethod
def add_parser_arguments(cls, parser):
parser.add_argument(
"-p",
"--python",
dest="python",
metavar="py",
help="target interpreter for which to create a virtual (either absolute path or identifier string)",
default=sys.executable,
)
def run(self):
return get_interpreter(self.python_spec)
def __str__(self):
return "{} discover of python_spec={!r}".format(self.__class__.__name__, self.python_spec)
def get_interpreter(key):
spec = PythonSpec.from_string_spec(key)
for interpreter, impl_must_match in propose_interpreters(spec):
if interpreter.satisfies(spec, impl_must_match):
return interpreter
def propose_interpreters(spec):
# 1. we always try with the lowest hanging fruit first, the current interpreter
yield CURRENT, True
# 2. if it's an absolut path and exists, use that
if spec.is_abs and os.path.exists(spec.path):
yield PythonInfo.from_exe(spec.path), True
# 3. otherwise fallback to platform default logic
if IS_WIN:
from .windows import propose_interpreters
for interpreter in propose_interpreters(spec):
yield interpreter, True
# 4. then maybe it's something exact on PATH - if it was direct lookup implementation no longer counts
interpreter = find_on_path(spec.str_spec)
if interpreter is not None:
yield interpreter, False
# 5. or from the spec we can deduce a name on path that matches
for exe, match in spec.generate_names():
interpreter = find_on_path(exe)
if interpreter is not None:
yield interpreter, match
def find_on_path(key):
exe = find_executable(key)
if exe is not None:
exe = os.path.abspath(exe)
interpreter = PythonInfo.from_exe(str(exe), raise_on_error=False)
return interpreter

27
src/virtualenv/interpreters/discovery/discover.py Normal file
View file

@ -0,0 +1,27 @@
from __future__ import absolute_import, unicode_literals
from abc import ABCMeta, abstractmethod
import six
@six.add_metaclass(ABCMeta)
class Discover(object):
def __init__(self):
self._has_run = False
self._interpreter = None
@classmethod
def add_parser_arguments(cls, parser):
raise NotImplementedError
@abstractmethod
def run(self):
raise NotImplementedError
@property
def interpreter(self):
if self._has_run is False:
self._interpreter = self.run()
self._has_run = True
return self._interpreter

215
src/virtualenv/interpreters/discovery/py_info.py Normal file
View file

@ -0,0 +1,215 @@
"""
The PythonInfo contains information about a concrete instance of a Python interpreter
Note: this file is also used to query target interpreters, so can only use standard library methods
"""
from __future__ import absolute_import, print_function, unicode_literals
import copy
import json
import logging
import os
import platform
import subprocess
import sys
from collections import OrderedDict, namedtuple
IS_WIN = sys.platform == "win32"
VersionInfo = namedtuple("VersionInfo", ["major", "minor", "micro", "releaselevel", "serial"])
def _get_path_extensions():
return list(OrderedDict.fromkeys([""] + os.environ.get("PATHEXT", "").lower().split(os.pathsep)))
EXTENSIONS = _get_path_extensions()
class PythonInfo(object):
"""Contains information for a Python interpreter"""
def __init__(self):
# qualifies the python
self.platform = sys.platform
self.implementation = platform.python_implementation()
# this is a tuple in earlier, struct later, unify to our own named tuple
self.version_info = VersionInfo(*list(sys.version_info))
self.architecture = 64 if sys.maxsize > 2 ** 32 else 32
self.executable = sys.executable # executable we were called with
self.original_executable = self.executable
self.base_executable = getattr(sys, "_base_executable", None) # some platforms may set this
self.version = sys.version
self.os = os.name
# information about the prefix - determines python home
self.prefix = getattr(sys, "prefix", None) # prefix we think
self.base_prefix = getattr(sys, "base_prefix", None) # venv
self.real_prefix = getattr(sys, "real_prefix", None) # old virtualenv
# information about the exec prefix - dynamic stdlib modules
self.base_exec_prefix = getattr(sys, "base_exec_prefix", None)
self.exec_prefix = getattr(sys, "exec_prefix", None)
try:
__import__("venv")
has = True
except ImportError:
has = False
self.has_venv = has
self.path = sys.path
@property
def version_str(self):
return ".".join(str(i) for i in self.version_info[0:3])
@property
def version_release_str(self):
return ".".join(str(i) for i in self.version_info[0:2])
@property
def python_name(self):
version_info = self.version_info
return "python{}.{}".format(version_info.major, version_info.minor)
@property
def is_old_virtualenv(self):
return self.real_prefix is not None
@property
def is_venv(self):
return self.base_prefix is not None and self.version_info.major == 3
def __repr__(self):
return "PythonInfo({!r})".format(self.__dict__)
def __str__(self):
content = copy.copy(self.__dict__)
for elem in ["path", "prefix", "base_prefix", "exec_prefix", "real_prefix", "base_exec_prefix"]:
del content[elem]
return "PythonInfo({!r})".format(content)
def to_json(self):
data = copy.deepcopy(self.__dict__)
# noinspection PyProtectedMember
data["version_info"] = data["version_info"]._asdict() # namedtuple to dictionary
return json.dumps(data, indent=2)
@classmethod
def from_json(cls, payload):
data = json.loads(payload)
data["version_info"] = VersionInfo(**data["version_info"]) # restore this to a named tuple structure
info = copy.deepcopy(CURRENT)
info.__dict__ = data
return info
@property
def system_prefix(self):
return self.real_prefix or self.base_prefix or self.prefix
@property
def system_exec_prefix(self):
return self.real_prefix or self.base_exec_prefix or self.exec_prefix
@property
def system_executable(self):
env_prefix = self.real_prefix or self.base_prefix
if env_prefix:
if self.real_prefix is None and self.base_executable is not None:
return self.base_executable
return self.find_exe(env_prefix)
else:
return self.executable
def find_exe(self, home):
# we don't know explicitly here, do some guess work - our executable name should tell
exe_base_name = os.path.basename(self.executable)
possible_names = self._find_possible_exe_names(exe_base_name)
possible_folders = self._find_possible_folders(exe_base_name, home)
for folder in possible_folders:
for name in possible_names:
candidate = os.path.join(folder, name)
if os.path.exists(candidate):
return candidate
what = "|".join(possible_names) # pragma: no cover
raise RuntimeError("failed to detect {} in {}".format(what, "|".join(possible_folders))) # pragma: no cover
def _find_possible_folders(self, exe_base_name, home):
candidate_folder = OrderedDict()
if self.executable.startswith(self.prefix):
relative = self.executable[len(self.prefix) : -len(exe_base_name)]
candidate_folder["{}{}".format(home, relative)] = None
candidate_folder[home] = None
return list(candidate_folder.keys())
@staticmethod
def _find_possible_exe_names(exe_base_name):
exe_no_suffix = os.path.splitext(exe_base_name)[0]
name_candidate = OrderedDict()
for ext in EXTENSIONS:
for at in range(3, -1, -1):
cur_ver = sys.version_info[0:at]
version = ".".join(str(i) for i in cur_ver)
name = "{}{}{}".format(exe_no_suffix, version, ext)
name_candidate[name] = None
return list(name_candidate.keys())
@classmethod
def from_exe(cls, exe, raise_on_error=True):
path = "{}.py".format(os.path.splitext(__file__)[0])
cmd = [exe, path]
# noinspection DuplicatedCode
# this is duplicated here because this file is executed on its own, so cannot be refactored otherwise
try:
process = subprocess.Popen(
cmd, universal_newlines=True, stdin=subprocess.PIPE, stderr=subprocess.PIPE, stdout=subprocess.PIPE
)
out, err = process.communicate()
code = process.returncode
except OSError as os_error:
out, err, code = "", os_error.strerror, os_error.errno
if code != 0:
if raise_on_error:
msg = "failed to query {} with code {}{}{}".format(
exe, code, " out: []".format(out) if out else "", " err: []".format(err) if err else ""
)
raise RuntimeError(msg)
else:
logging.debug("failed %s with code %s out %s err %s", cmd, code, out, err)
return None
result = cls.from_json(out)
result.executable = exe # keep original executable as this may contain initialization code
return result
def satisfies(self, spec, impl_must_match):
"""check if a given specification can be satisfied by the this python interpreter instance"""
if self.executable == spec.path: # if the path is a our own executable path we're done
return True
if spec.path is not None: # if path set, and is not our original executable name, this does not match
root, _ = os.path.splitext(os.path.basename(self.original_executable))
if root != spec.path:
return False
if impl_must_match:
if spec.implementation is not None and spec.implementation != self.implementation:
return False
if spec.architecture is not None and spec.architecture != self.architecture:
return False
for our, req in zip(self.version_info[0:3], (spec.major, spec.minor, spec.patch)):
if req is not None and our is not None and our != req:
return False
return True
CURRENT = PythonInfo()
if __name__ == "__main__":
print(CURRENT.to_json())

115
src/virtualenv/interpreters/discovery/py_spec.py Normal file
View file

@ -0,0 +1,115 @@
"""A Python specification is an abstract requirement definition of a interpreter"""
from __future__ import absolute_import, unicode_literals
import os
import re
import sys
from collections import OrderedDict
PATTERN = re.compile(r"^(?P<impl>[a-zA-Z]+)(?P<version>[0-9.]+)?(?:-(?P<arch>32|64))?$")
IS_WIN = sys.platform == "win32"
class PythonSpec(object):
"""Contains specification about a Python Interpreter"""
def __init__(self, str_spec, implementation, major, minor, patch, architecture, path):
self.str_spec = str_spec
self.implementation = implementation
self.major = major
self.minor = minor
self.patch = patch
self.architecture = architecture
self.path = path
@classmethod
def from_string_spec(cls, string_spec):
impl, major, minor, patch, arch, path = None, None, None, None, None, None
if os.path.isabs(string_spec):
path = string_spec
else:
ok = False
match = re.match(PATTERN, string_spec)
if match:
def _int_or_none(val):
return None if val is None else int(val)
try:
groups = match.groupdict()
version = groups["version"]
if version is not None:
versions = tuple(int(i) for i in version.split(".") if i)
if len(versions) > 3:
raise ValueError
if len(versions) == 3:
major, minor, patch = versions
elif len(versions) == 2:
major, minor = versions
elif len(versions) == 1:
version_data = versions[0]
major = int(str(version_data)[0]) # first digit major
if version_data > 9:
minor = int(str(version_data)[1:])
ok = True
except ValueError:
pass
else:
impl = groups["impl"]
if impl == "py" or impl == "python":
impl = "CPython"
arch = _int_or_none(groups["arch"])
if not ok:
path = string_spec
return cls(string_spec, impl, major, minor, patch, arch, path)
def generate_names(self):
impls = OrderedDict()
if self.implementation:
# first consider implementation as it is
impls[self.implementation] = False
# for case sensitive file systems consider lower and upper case versions too
# trivia: MacBooks and all pre 2018 Windows-es were case insensitive by default
impls[self.implementation.lower()] = False
impls[self.implementation.upper()] = False
impls["python"] = True # finally consider python as alias, implementation must match now
version = self.major, self.minor, self.patch
try:
version = version[: version.index(None)]
except ValueError:
pass
for impl, match in impls.items():
for at in range(len(version), -1, -1):
cur_ver = version[0:at]
spec = "{}{}".format(impl, ".".join(str(i) for i in cur_ver))
yield spec, match
@property
def is_abs(self):
return self.path is not None and os.path.isabs(self.path)
def satisfies(self, spec):
"""called when there's a candidate metadata spec to see if compatible - e.g. PEP-514 on Windows"""
if spec.is_abs and self.is_abs and self.path != spec.path:
return False
if spec.implementation is not None and spec.implementation != self.implementation:
return False
if spec.architecture is not None and spec.architecture != self.architecture:
return False
for our, req in zip((self.major, self.minor, self.patch), (spec.major, spec.minor, spec.patch)):
if req is not None and our is not None and our != req:
return False
return True
def __repr__(self):
return "{}({})".format(
type(self).__name__,
", ".join(
"{}={}".format(k, getattr(self, k))
for k in ("str_spec", "implementation", "major", "minor", "patch", "architecture", "path")
if getattr(self, k) is not None
),
)

17
src/virtualenv/interpreters/discovery/windows/__init__.py Normal file
View file

@ -0,0 +1,17 @@
from __future__ import absolute_import, unicode_literals
from ..py_info import PythonInfo
from ..py_spec import PythonSpec
from .pep514 import discover_pythons
def propose_interpreters(spec):
# see if PEP-514 entries are good
for name, major, minor, arch, exe, _ in discover_pythons():
# pre-filter
registry_spec = PythonSpec(None, name, major, minor, None, arch, exe)
if registry_spec.satisfies(spec):
interpreter = PythonInfo.from_exe(exe, raise_on_error=False)
if interpreter is not None:
if interpreter.satisfies(spec, impl_must_match=True):
yield interpreter

162
src/virtualenv/interpreters/discovery/windows/pep514.py Normal file
View file

@ -0,0 +1,162 @@
"""Implement https://www.python.org/dev/peps/pep-0514/ to discover interpreters - Windows only"""
from __future__ import absolute_import, print_function, unicode_literals
import os
import re
from logging import basicConfig, getLogger
import six
if six.PY3:
import winreg
else:
# noinspection PyUnresolvedReferences
import _winreg as winreg
LOGGER = getLogger(__name__)
def enum_keys(key):
at = 0
while True:
try:
yield winreg.EnumKey(key, at)
except OSError:
break
at += 1
def get_value(key, value_name):
try:
return winreg.QueryValueEx(key, value_name)[0]
except OSError:
return None
def discover_pythons():
for hive, hive_name, key, flags, default_arch in [
(winreg.HKEY_CURRENT_USER, "HKEY_CURRENT_USER", r"Software\Python", 0, 64),
(winreg.HKEY_LOCAL_MACHINE, "HKEY_LOCAL_MACHINE", r"Software\Python", winreg.KEY_WOW64_64KEY, 64),
(winreg.HKEY_LOCAL_MACHINE, "HKEY_LOCAL_MACHINE", r"Software\Python", winreg.KEY_WOW64_32KEY, 32),
]:
for spec in process_set(hive, hive_name, key, flags, default_arch):
yield spec
def process_set(hive, hive_name, key, flags, default_arch):
try:
with winreg.OpenKeyEx(hive, key, 0, winreg.KEY_READ | flags) as root_key:
for company in enum_keys(root_key):
if company == "PyLauncher": # reserved
continue
for spec in process_company(hive_name, company, root_key, default_arch):
yield spec
except OSError:
pass
def process_company(hive_name, company, root_key, default_arch):
with winreg.OpenKeyEx(root_key, company) as company_key:
for tag in enum_keys(company_key):
spec = process_tag(hive_name, company, company_key, tag, default_arch)
if spec is not None:
yield spec
def process_tag(hive_name, company, company_key, tag, default_arch):
with winreg.OpenKeyEx(company_key, tag) as tag_key:
version = load_version_data(hive_name, company, tag, tag_key)
if version is not None: # if failed to get version bail
major, minor, _ = version
arch = load_arch_data(hive_name, company, tag, tag_key, default_arch)
if arch is not None:
exe_data = load_exe(hive_name, company, company_key, tag)
if exe_data is not None:
exe, args = exe_data
name = str("python") if company == "PythonCore" else company
return name, major, minor, arch, exe, args
def load_exe(hive_name, company, company_key, tag):
key_path = "{}/{}/{}".format(hive_name, company, tag)
try:
with winreg.OpenKeyEx(company_key, r"{}\InstallPath".format(tag)) as ip_key:
with ip_key:
exe = get_value(ip_key, "ExecutablePath")
if exe is None:
ip = get_value(ip_key, None)
if ip is None:
msg(key_path, "no ExecutablePath or default for it")
else:
exe = os.path.join(ip, str("python.exe"))
if exe is not None and os.path.exists(exe):
args = get_value(ip_key, "ExecutableArguments")
return exe, args
else:
msg(key_path, "exe does not exists {}".format(key_path, exe))
except OSError:
msg("{}/{}".format(key_path, "InstallPath"), "missing")
return None
def load_arch_data(hive_name, company, tag, tag_key, default_arch):
arch_str = get_value(tag_key, "SysArchitecture")
if arch_str is not None:
key_path = "{}/{}/{}/SysArchitecture".format(hive_name, company, tag)
try:
return parse_arch(arch_str)
except ValueError as sys_arch:
msg(key_path, sys_arch)
return default_arch
def parse_arch(arch_str):
if isinstance(arch_str, six.string_types):
match = re.match(r"^(\d+)bit$", arch_str)
if match:
return int(next(iter(match.groups())))
error = "invalid format {}".format(arch_str)
else:
error = "arch is not string: {}".format(repr(arch_str))
raise ValueError(error)
def load_version_data(hive_name, company, tag, tag_key):
for candidate, key_path in [
(get_value(tag_key, "SysVersion"), "{}/{}/{}/SysVersion".format(hive_name, company, tag)),
(tag, "{}/{}/{}".format(hive_name, company, tag)),
]:
if candidate is not None:
try:
return parse_version(candidate)
except ValueError as sys_version:
msg(key_path, sys_version)
return None
def parse_version(version_str):
if isinstance(version_str, six.string_types):
match = re.match(r"^(\d+)(?:\.(\d+))?(?:\.(\d+))?$", version_str)
if match:
return tuple(int(i) if i is not None else None for i in match.groups())
error = "invalid format {}".format(version_str)
else:
error = "version is not string: {}".format(repr(version_str))
raise ValueError(error)
def msg(path, what):
LOGGER.warning("PEP-514 violation in Windows Registry at {} error: {}".format(path, what))
def _run():
basicConfig()
interpreters = []
for spec in discover_pythons():
interpreters.append(repr(spec))
print("\n".join(sorted(interpreters)))
if __name__ == "__main__":
_run()

54
src/virtualenv/pyenv_cfg.py Normal file
View file

@ -0,0 +1,54 @@
from __future__ import absolute_import, unicode_literals
import logging
class PyEnvCfg(object):
def __init__(self, content, path):
self.content = content
self.path = path
@classmethod
def from_folder(cls, folder):
return cls.from_file(folder / "pyvenv.cfg")
@classmethod
def from_file(cls, path):
content = cls._read_values(path) if path.exists() else {}
return PyEnvCfg(content, path)
@staticmethod
def _read_values(path):
content = {}
for line in path.read_text().splitlines():
equals_at = line.index("=")
key = line[:equals_at].strip()
value = line[equals_at + 1 :].strip()
content[key] = value
return content
def write(self):
with open(str(self.path), "wt") as file_handler:
logging.debug("write %s", self.path)
for key, value in self.content.items():
line = "{} = {}".format(key, value)
logging.debug("\t%s", line)
file_handler.write(line)
file_handler.write("\n")
def refresh(self):
self.content = self._read_values(self.path)
return self.content
def __setitem__(self, key, value):
self.content[key] = value
def __getitem__(self, key):
return self.content[key]
def __contains__(self, item):
return item in self.content
def update(self, other):
self.content.update(other)
return self

43
src/virtualenv/report.py Normal file
View file

@ -0,0 +1,43 @@
from __future__ import absolute_import, unicode_literals
import logging
import sys
LEVELS = {
0: logging.CRITICAL,
1: logging.ERROR,
2: logging.WARNING,
3: logging.INFO,
4: logging.DEBUG,
5: logging.NOTSET,
}
MAX_LEVEL = max(LEVELS.keys())
LOGGER = logging.getLogger()
def setup_report(verbose, quiet):
verbosity = max(verbose - quiet, 0)
_clean_handlers(LOGGER)
if verbosity > MAX_LEVEL:
verbosity = MAX_LEVEL # pragma: no cover
level = LEVELS[verbosity]
msg_format = "%(message)s"
if level >= logging.DEBUG:
locate = "pathname" if level > logging.DEBUG else "module"
msg_format += "[%(asctime)s] %(levelname)s [%({})s:%(lineno)d]".format(locate)
formatter = logging.Formatter(str(msg_format))
stream_handler = logging.StreamHandler(stream=sys.stdout)
stream_handler.setLevel(level)
LOGGER.setLevel(logging.NOTSET)
stream_handler.setFormatter(formatter)
LOGGER.addHandler(stream_handler)
level_name = logging.getLevelName(level)
logging.debug("setup logging to %s", level_name)
return verbosity
def _clean_handlers(log):
for log_handler in list(log.handlers): # remove handlers of libraries
log.removeHandler(log_handler)

166
src/virtualenv/run.py Normal file
View file

@ -0,0 +1,166 @@
from __future__ import absolute_import, unicode_literals
import logging
from entrypoints import get_group_named
from .config.cli.parser import VirtualEnvConfigParser
from .report import LEVELS, setup_report
from .session import Session
def run_via_cli(args):
"""Run the virtual environment creation via CLI arguments
:param args: the command line arguments
:return: the creator used
"""
session = session_via_cli(args)
session.run()
return session
def session_via_cli(args):
parser = VirtualEnvConfigParser()
options, verbosity = _do_report_setup(parser, args)
discover = _get_discover(parser, args, options)
interpreter = discover.interpreter
if interpreter is None:
raise RuntimeError("failed to find interpreter for {}".format(discover))
elements = [
_get_creator(interpreter, parser, options),
_get_seeder(parser, options),
_get_activation(interpreter, parser, options),
]
[next(elem) for elem in elements] # add choice of types
parser.parse_known_args(args, namespace=options)
[next(elem) for elem in elements] # add type flags
parser.enable_help()
parser.parse_args(args, namespace=options)
creator, seeder, activators = tuple(next(e) for e in elements) # create types
session = Session(verbosity, interpreter, creator, seeder, activators)
return session
def _do_report_setup(parser, args):
level_map = ", ".join("{}:{}".format(c, logging.getLevelName(l)) for c, l in sorted(list(LEVELS.items())))
msg = "verbosity = verbose - quiet, default {}, count mapping = {{{}}}"
verbosity_group = parser.add_argument_group(msg.format(logging.getLevelName(LEVELS[3]), level_map))
verbosity = verbosity_group.add_mutually_exclusive_group()
verbosity.add_argument("-v", "--verbose", action="count", dest="verbose", help="increase verbosity", default=3)
verbosity.add_argument("-q", "--quiet", action="count", dest="quiet", help="decrease verbosity", default=0)
options, _ = parser.parse_known_args(args)
verbosity_value = setup_report(options.verbose, options.quiet)
return options, verbosity_value
def _get_discover(parser, args, options):
discover_types = _collect_discovery_types()
discovery_parser = parser.add_argument_group("target interpreter identifier")
discovery_parser.add_argument(
"--discovery",
choices=list(discover_types.keys()),
default=next(i for i in discover_types.keys()),
required=False,
help="interpreter discovery method",
)
options, _ = parser.parse_known_args(args, namespace=options)
discover_class = discover_types[options.discovery]
discover_class.add_parser_arguments(discovery_parser)
options, _ = parser.parse_known_args(args, namespace=options)
discover = discover_class(options)
return discover
def _collect_discovery_types():
discover_types = {e.name: e.load() for e in get_group_named("virtualenv.discovery").values()}
return discover_types
def _get_creator(interpreter, parser, options):
creators = _collect_creators(interpreter)
creator_parser = parser.add_argument_group("creator options")
creator_parser.add_argument(
"--creator",
choices=list(creators.keys()),
default=next((c for c in creators if c != "venv"), None),
required=False,
help="create environment via",
)
yield
if options.creator not in creators:
raise RuntimeError("No virtualenv implementation for {}".format(interpreter))
creator_class = creators[options.creator]
creator_class.add_parser_arguments(creator_parser, interpreter)
yield
creator = creator_class(options, interpreter)
yield creator
def _collect_creators(interpreter):
all_creators = {e.name: e.load() for e in get_group_named("virtualenv.create").values()}
creators = {k: v for k, v in all_creators.items() if v.supports(interpreter)}
return creators
def _get_seeder(parser, options):
seed_parser = parser.add_argument_group("package seeder")
seeder_types = _collect_seeders()
seed_parser.add_argument(
"--seeder",
choices=list(seeder_types.keys()),
default="link-app-data",
required=False,
help="seed packages install method",
)
seed_parser.add_argument(
"--without-pip",
help="if set forces the none seeder, used for compatibility with venv",
action="store_true",
dest="without_pip",
)
yield
seeder_class = seeder_types["none" if options.without_pip is True else options.seeder]
seeder_class.add_parser_arguments(seed_parser)
yield
seeder = seeder_class(options)
yield seeder
def _collect_seeders():
seeder_types = {e.name: e.load() for e in get_group_named("virtualenv.seed").values()}
return seeder_types
def _get_activation(interpreter, parser, options):
activator_parser = parser.add_argument_group("activation script generator")
activators = _collect_activators(interpreter)
activator_parser.add_argument(
"--activators",
choices=list(activators.keys()),
default=list(activators.keys()),
required=False,
nargs="*",
help="activators to generate",
)
yield
active_activators = {k: v for k, v in activators.items() if k in options.activators}
activator_parser.add_argument(
"--prompt",
dest="prompt",
metavar="prompt",
help="provides an alternative prompt prefix for this environment",
default=None,
)
for activator in active_activators.values():
activator.add_parser_arguments(parser)
yield
activator_instances = [activator_class(options) for activator_class in active_activators.values()]
yield activator_instances
def _collect_activators(interpreter):
all_activators = {e.name: e.load() for e in get_group_named("virtualenv.activate").values()}
activators = {k: v for k, v in all_activators.items() if v.supports(interpreter)}
return activators

1
src/virtualenv/seed/__init__.py Normal file
View file

@ -0,0 +1 @@
from __future__ import absolute_import, unicode_literals

1
src/virtualenv/seed/embed/__init__.py Normal file
View file

@ -0,0 +1 @@
from __future__ import absolute_import, unicode_literals

44
src/virtualenv/seed/embed/base_embed.py Normal file
View file

@ -0,0 +1,44 @@
from __future__ import absolute_import, unicode_literals
from abc import ABCMeta
import six
from ..seeder import Seeder
@six.add_metaclass(ABCMeta)
class BaseEmbed(Seeder):
def __init__(self, options):
super(Seeder, self).__init__()
self.enabled = options.without_pip is False
self.download = options.download
self.pip_version = options.pip
self.setuptools_version = options.setuptools
@classmethod
def add_parser_arguments(cls, parser):
group = parser.add_mutually_exclusive_group()
group.add_argument(
"--download",
dest="download",
action="store_true",
help="download latest pip/setuptools from PyPi",
default=False,
)
group.add_argument(
"--no-download",
dest="download",
action="store_false",
help="do not download latest pip/setuptools from PyPi",
default=True,
)
for package in ["pip", "setuptools"]:
parser.add_argument(
"--{}".format(package),
dest=package,
metavar="version",
help="{} version to install, default: latest from cache, bundle for bundled".format(package),
default=None,
)

217
src/virtualenv/seed/embed/link_app_data.py Normal file
View file

@ -0,0 +1,217 @@
"""Bootstrap"""
from __future__ import absolute_import, unicode_literals
import logging
import os
import shutil
import sys
import zipfile
from shutil import copytree
from textwrap import dedent
from pathlib2 import Path
from six import PY3
from virtualenv.info import get_default_data_dir
from .base_embed import BaseEmbed
from .wheels.acquire import get_wheel
try:
import ConfigParser
except ImportError:
# noinspection PyPep8Naming
import configparser as ConfigParser
class LinkFromAppData(BaseEmbed):
def run(self, creator):
if not self.enabled:
return
cache = get_default_data_dir() / "seed-v1"
version = creator.interpreter.version_release_str
name_to_whl = get_wheel(version, cache, self.download, self.pip_version, self.setuptools_version)
pip_install(name_to_whl, creator, cache)
def pip_install(wheels, creator, cache):
site_package, bin_dir, env_exe = creator.site_packages[0], creator.bin_dir, creator.env_exe
folder_link_method, folder_linked = link_folder()
for name, wheel in wheels.items():
logging.debug("install %s from wheel %s", name, wheel)
extracted = _get_extracted(cache, wheel)
added, dist_info = _link_content(folder_link_method, site_package, extracted)
extra_files = _generate_extra_files(bin_dir, env_exe, site_package, dist_info, creator)
fix_records(creator, dist_info, site_package, folder_linked, added, extra_files)
def link_folder():
if sys.platform == "win32":
# on Windows symlinks are unreliable, but we have junctions for folders
# sadly junctions don't play well with pip yet as the remove cleans the target
# if sys.version_info[0:2] > (3, 4):
# import _winapi # python3.5 has builtin implementation for junctions
#
# return _winapi.CreateJunction, True
pass
else:
# on POSIX prefer symlinks, however symlink support requires pip 19.3+, not supported by pip
if sys.version_info[0:2] != (3, 4):
return os.symlink, True
# if nothing better fallback to copy the tree
return copytree, False
def _get_extracted(cache, wheel):
install_image = cache / "extract"
install_image.mkdir(parents=True, exist_ok=True)
extracted = install_image / wheel.name
if not extracted.exists():
logging.debug("create install image to %s of %s", extracted, wheel)
extracted.mkdir(parents=True, exist_ok=True)
with zipfile.ZipFile(str(wheel)) as zip_ref:
zip_ref.extractall(str(extracted))
else:
logging.debug("install from extract %s", extracted)
return extracted
def _link_content(folder_link, site_package, extracted):
added = []
dist_info = None
for filename in extracted.iterdir():
into = site_package / filename.name
if into.exists():
if into.is_dir() and not into.is_symlink():
shutil.rmtree(str(into))
else:
into.unlink()
is_dir = filename.is_dir()
method = folder_link if is_dir else shutil.copy2
method(str(filename), str(into))
added.append((is_dir, into))
if filename.suffix == ".dist-info":
dist_info = into
return added, dist_info
def _generate_extra_files(bin_dir, env_exe, site_package, dist_info, creator):
extra = []
# pretend was installed by pip
installer = dist_info / "INSTALLER"
installer.write_text("pip\n")
extra.append(installer)
# inject a no-op root element, as workaround for bug added by https://github.com/pypa/pip/commit/c7ae06c79#r35523722
marker = site_package / "{}.virtualenv".format(dist_info.name)
marker.write_text("")
extra.append(marker)
console_scripts = load_console_scripts(dist_info, creator)
for name, value in console_scripts:
extra.extend(create_console_entry_point(bin_dir, name, value, env_exe, creator))
return extra
def load_console_scripts(dist_info, creator):
result = []
entry_points = dist_info / "entry_points.txt"
if entry_points.exists():
parser = ConfigParser.ConfigParser()
with entry_points.open() as file_handler:
reader = getattr(parser, "read_file" if PY3 else "readfp")
reader(file_handler)
if "console_scripts" in parser.sections():
for name, value in parser.items("console_scripts"):
result.append((name, value))
return result
def create_console_entry_point(bin_dir, name, value, env_exe, creator):
result = []
if sys.platform == "win32":
# windows doesn't support simple script files, so fallback to more complicated exe generator
from distlib.scripts import ScriptMaker
maker = ScriptMaker(None, str(bin_dir))
maker.clobber = True # overwrite
maker.variants = {"", "X", "X.Y"} # create all variants
maker.set_mode = True # ensure they are executable
maker.executable = str(env_exe)
specification = "{} = {}".format(name, value)
new_files = maker.make(specification)
result.extend(new_files)
else:
module, func = value.split(":")
content = (
dedent(
"""
#!{0}
# -*- coding: utf-8 -*-
import re
import sys
from {1} import {2}
if __name__ == "__main__":
sys.argv[0] = re.sub(r"(-script.pyw?|.exe)?$", "", sys.argv[0])
sys.exit({2}())
"""
)
.lstrip()
.format(env_exe, module, func)
)
version = creator.interpreter.version_info
for new_name in (
name,
"{}{}".format(name, version.major),
"{}-{}.{}".format(name, version.major, version.minor),
):
exe = bin_dir / new_name
exe.write_text(content)
exe.chmod(0o755)
result.append(exe)
return result
def fix_records(creator, dist_info, site_package, folder_linked, added, extra_files):
extra_records = []
version = creator.interpreter.version_info
py_c_ext = ".{}-{}{}.pyc".format(creator.interpreter.implementation.lower(), version.major, version.minor)
def _handle_file(of, base):
if of.suffix == ".py":
pyc = base / "{}{}".format(of.stem, py_c_ext)
extra_records.append(pyc)
for is_dir, file in added:
if is_dir:
if folder_linked:
extra_records.append(file)
else:
for root, _, filenames in os.walk(str(file)):
root_path = Path(root) / "__pycache__"
for filename in filenames:
_handle_file(Path(filename), root_path)
else:
root_path = file.parent / "__pycache__"
_handle_file(file, root_path)
extra_records.append(file)
extra_records.extend(extra_files)
new_records = []
for rec in extra_records:
name = os.path.relpath(str(rec), str(site_package))
new_records.append("{},,".format(name))
record = dist_info / "RECORD"
content = ("" if folder_linked else record.read_text()) + "\n".join(new_records)
record.write_text(content)
def add_record_line(name):
return "{},,".format(name)

12
src/virtualenv/seed/embed/pip_invoke.py Normal file
View file

@ -0,0 +1,12 @@
from __future__ import absolute_import, unicode_literals
from .base_embed import BaseEmbed
class PipInvoke(BaseEmbed):
def __init__(self, options):
super(PipInvoke, self).__init__(options)
def run(self, creator):
if not self.enabled:
return

12
src/virtualenv/seed/embed/wheels/__init__.py Normal file
View file

@ -0,0 +1,12 @@
from __future__ import absolute_import, unicode_literals
BUNDLE_SUPPORT = {
"3.9": {"pip": "pip-19.3.1-py2.py3-none-any.whl", "setuptools": "setuptools-41.6.0-py2.py3-none-any.whl"},
"3.8": {"pip": "pip-19.3.1-py2.py3-none-any.whl", "setuptools": "setuptools-41.6.0-py2.py3-none-any.whl"},
"3.7": {"pip": "pip-19.3.1-py2.py3-none-any.whl", "setuptools": "setuptools-41.6.0-py2.py3-none-any.whl"},
"3.6": {"pip": "pip-19.3.1-py2.py3-none-any.whl", "setuptools": "setuptools-41.6.0-py2.py3-none-any.whl"},
"3.5": {"pip": "pip-19.3.1-py2.py3-none-any.whl", "setuptools": "setuptools-41.6.0-py2.py3-none-any.whl"},
"3.4": {"pip": "pip-19.1.1-py2.py3-none-any.whl", "setuptools": "setuptools-41.6.0-py2.py3-none-any.whl"},
"2.7": {"pip": "pip-19.3.1-py2.py3-none-any.whl", "setuptools": "setuptools-41.6.0-py2.py3-none-any.whl"},
}
MAX = "3.9"

94
src/virtualenv/seed/embed/wheels/acquire.py Normal file
View file

@ -0,0 +1,94 @@
"""Bootstrap"""
from __future__ import absolute_import, unicode_literals
from collections import defaultdict
from shutil import copy2
from pathlib2 import Path
from . import BUNDLE_SUPPORT, MAX
BUNDLE_FOLDER = Path(__file__).parent
def get_wheel(version_release, cache, download, pip, setuptools):
wheel_download = cache / "download" / version_release
wheel_download.mkdir(parents=True, exist_ok=True)
packages = {"pip": pip, "setuptools": setuptools}
ensure_bundle_cached(
packages, version_release, wheel_download
) # first ensure all bundled versions area already there
if download is True:
must_download = check_if_must_download(packages, wheel_download) # check what needs downloading
if must_download: # perform download if any of the packages require
download_wheel(version_release, must_download, wheel_download)
return _get_wheels_for_package(wheel_download, packages)
def download_wheel(version_str, must_download, wheel_download):
cmd = [
"download",
"--disable-pip-version-check",
"--only-binary=:all:",
"--no-deps",
"--python-version",
version_str,
"-d",
str(wheel_download),
]
cmd.extend(must_download)
from pip._internal import main
main(cmd)
def check_if_must_download(packages, wheel_download):
must_download = set()
if any(i is not None for i in packages.values()):
has_version = _get_wheels(wheel_download)
for pkg, version in packages.items():
if pkg in has_version and version in has_version[pkg]:
continue
must_download.add(pkg)
return must_download
def _get_wheels(inside_folder):
has_version = defaultdict(set)
for filename in inside_folder.iterdir():
if filename.suffix == ".whl":
pkg, version = filename.stem.split("-")[0:2]
has_version[pkg].add(version)
return has_version
def _get_wheels_for_package(inside_folder, package):
has_version = defaultdict(dict)
for filename in inside_folder.iterdir():
if filename.suffix == ".whl":
pkg, version = filename.stem.split("-")[0:2]
has_version[pkg][version] = filename
result = {}
for pkg, version in package.items():
content = has_version[pkg]
if version in content:
result[pkg] = content[version]
else:
elements = sorted(
content.items(),
key=lambda a: tuple(int(i) if i.isdigit() else i for i in a[0].split(".")),
reverse=True,
)
result[pkg] = elements[0][1]
return result
def ensure_bundle_cached(packages, version_release, wheel_download):
for package in packages:
bundle = (BUNDLE_SUPPORT.get(version_release, {}) or BUNDLE_SUPPORT[MAX]).get(package)
if bundle is not None:
bundled_wheel_file = wheel_download / bundle
if not bundled_wheel_file.exists():
copy2(str(BUNDLE_FOLDER / bundle), str(bundled_wheel_file))

BIN
src/virtualenv/seed/embed/wheels/pip-19.1.1-py2.py3-none-any.whl Normal file
View file

Binary file not shown.

BIN
src/virtualenv/seed/embed/wheels/pip-19.3.1-py2.py3-none-any.whl Normal file
View file

Binary file not shown.

BIN
src/virtualenv/seed/embed/wheels/setuptools-41.6.0-py2.py3-none-any.whl Normal file
View file

Binary file not shown.

12
src/virtualenv/seed/none.py Normal file
View file

@ -0,0 +1,12 @@
from __future__ import absolute_import, unicode_literals
from virtualenv.seed.seeder import Seeder
class NoneSeeder(Seeder):
@classmethod
def add_parser_arguments(cls, parser):
pass
def run(self, creator):
pass

19
src/virtualenv/seed/seeder.py Normal file
View file

@ -0,0 +1,19 @@
from __future__ import absolute_import, unicode_literals
from abc import ABCMeta, abstractmethod
import six
@six.add_metaclass(ABCMeta)
class Seeder(object):
def __init__(self, options):
pass
@classmethod
def add_parser_arguments(cls, parser):
raise NotImplementedError
@abstractmethod
def run(self, creator):
raise NotImplementedError

45
src/virtualenv/session.py Normal file
View file

@ -0,0 +1,45 @@
from __future__ import absolute_import, unicode_literals
import json
import logging
class Session(object):
def __init__(self, verbosity, interpreter, creator, seeder, activators):
self.verbosity = verbosity
self.interpreter = interpreter
self.creator = creator
self.seeder = seeder
self.activators = activators
def run(self):
self._create()
self._seed()
self._activate()
self.creator.pyenv_cfg.write()
def _create(self):
self.creator.run()
logging.debug(_DEBUG_MARKER)
logging.debug("%s", _Debug(self.creator))
def _seed(self):
if self.seeder is not None:
self.seeder.run(self.creator)
def _activate(self):
for activator in self.activators:
activator.run(self.creator)
_DEBUG_MARKER = "=" * 30 + " target debug " + "=" * 30
class _Debug(object):
"""lazily populate debug"""
def __init__(self, creator):
self.creator = creator
def __str__(self):
return json.dumps(self.creator.debug, indent=2)

58
src/virtualenv/util.py Normal file
View file

@ -0,0 +1,58 @@
from __future__ import absolute_import, unicode_literals
import logging
import os
import shutil
import subprocess
from functools import partial
from os import makedirs
import six
def ensure_dir(path):
if not path.exists():
logging.debug("created %s", path)
makedirs(six.text_type(path))
HAS_SYMLINK = hasattr(os, "symlink")
def symlink_or_copy(do_copy, src, dst, relative_symlinks_ok=False):
"""
Try symlinking a target, and if that fails, fall back to copying.
"""
if do_copy is False and HAS_SYMLINK is False: # if no symlink, always use copy
do_copy = True
if not do_copy:
try:
if not dst.is_symlink(): # can't link to itself!
if relative_symlinks_ok:
assert src.parent == dst.parent
os.symlink(src.name, six.text_type(dst))
else:
os.symlink(six.text_type(src), six.text_type(dst))
except OSError as exception:
logging.warning("symlink failed %r, for %s to %s, will try copy", exception, src, dst)
do_copy = True
if do_copy:
copier = shutil.copy2 if src.is_file() else shutil.copytree
copier(six.text_type(src), six.text_type(dst))
logging.debug("%s %s to %s", "copy" if do_copy else "symlink", src, dst)
def run_cmd(cmd):
try:
process = subprocess.Popen(
cmd, universal_newlines=True, stdin=subprocess.PIPE, stderr=subprocess.PIPE, stdout=subprocess.PIPE
)
out, err = process.communicate() # input disabled
code = process.returncode
except OSError as os_error:
code, out, err = os_error.errno, "", os_error.strerror
return code, out, err
symlink = partial(symlink_or_copy, False)
copy = partial(symlink_or_copy, True)

38
tasks/make_zipapp.py Normal file
View file

@ -0,0 +1,38 @@
"""https://docs.python.org/3/library/zipapp.html"""
import argparse
import io
import os.path
import zipapp
import zipfile
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--root", default=".")
parser.add_argument("--dest")
args = parser.parse_args()
if args.dest is not None:
dest = args.dest
else:
dest = os.path.join(args.root, "virtualenv.pyz")
filenames = {"LICENSE.txt": "LICENSE.txt", os.path.join("src", "virtualenv.py"): "virtualenv.py"}
for support in os.listdir(os.path.join(args.root, "src", "virtualenv_support")):
support_file = os.path.join("virtualenv_support", support)
filenames[os.path.join("src", support_file)] = support_file
bio = io.BytesIO()
with zipfile.ZipFile(bio, "w") as zip_file:
for filename in filenames:
zip_file.write(os.path.join(args.root, filename), filename)
zip_file.writestr("__main__.py", "import virtualenv; virtualenv.main()")
bio.seek(0)
zipapp.create_archive(bio, dest)
print("zipapp created at {}".format(dest))
if __name__ == "__main__":
exit(main())

76
tasks/release.py Normal file
View file

@ -0,0 +1,76 @@
# -*- coding: utf-8 -*-
"""Handles creating a release PR"""
from pathlib import Path
from subprocess import check_call
from typing import Tuple
from git import Commit, Head, Remote, Repo, TagReference
from packaging.version import Version
ROOT_SRC_DIR = Path(__file__).resolve().parents[1]
def main(version_str: str) -> None:
version = Version(version_str)
repo = Repo(str(ROOT_SRC_DIR))
if repo.is_dirty():
raise RuntimeError("Current repository is dirty. Please commit any changes and try again.")
upstream, release_branch = create_release_branch(repo, version)
release_commit = release_changelog(repo, version)
tag = tag_release_commit(release_commit, repo, version)
print("push release commit")
repo.git.push(upstream.name, release_branch)
print("push release tag")
repo.git.push(upstream.name, tag)
print("All done! ✨ 🍰 ✨")
def create_release_branch(repo: Repo, version: Version) -> Tuple[Remote, Head]:
print("create release branch from upstream master")
upstream = get_upstream(repo)
upstream.fetch()
branch_name = f"release-{version}"
release_branch = repo.create_head(branch_name, upstream.refs.master, force=True)
upstream.push(refspec=f"{branch_name}:{branch_name}", force=True)
release_branch.set_tracking_branch(repo.refs[f"{upstream.name}/{branch_name}"])
release_branch.checkout()
return upstream, release_branch
def get_upstream(repo: Repo) -> Remote:
upstream_remote = "pypa/virtualenv.git"
urls = set()
for remote in repo.remotes:
for url in remote.urls:
if url.endswith(upstream_remote):
return remote
urls.add(url)
raise RuntimeError(f"could not find {upstream_remote} remote, has {urls}")
def release_changelog(repo: Repo, version: Version) -> Commit:
print("generate release commit")
check_call(["towncrier", "--yes", "--version", version.public], cwd=str(ROOT_SRC_DIR))
release_commit = repo.index.commit(f"release {version}")
return release_commit
def tag_release_commit(release_commit, repo, version) -> TagReference:
print("tag release commit")
existing_tags = [x.name for x in repo.tags]
if version in existing_tags:
print("delete existing tag {}".format(version))
repo.delete_tag(version)
print("create tag {}".format(version))
tag = repo.create_tag(version, ref=release_commit, force=True)
return tag
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser(prog="release")
parser.add_argument("--version", required=True)
options = parser.parse_args()
main(options.version)

89
tasks/update_embedded.py Executable file
View file

@ -0,0 +1,89 @@
#!/usr/bin/env python
"""
Helper script to rebuild virtualenv.py from virtualenv_support
"""
from __future__ import print_function, unicode_literals
import codecs
import os
import re
import sys
from zlib import crc32 as _crc32
if sys.version_info < (3,):
print("requires Python 3 (use tox from Python 3 if invoked via tox)")
raise SystemExit(1)
def crc32(data):
"""Python version idempotent"""
return _crc32(data.encode()) & 0xFFFFFFFF
here = os.path.realpath(os.path.dirname(__file__))
script = os.path.realpath(os.path.join(here, "..", "src", "virtualenv.py"))
gzip = codecs.lookup("zlib")
b64 = codecs.lookup("base64")
file_regex = re.compile(r'# file (.*?)\n([a-zA-Z][a-zA-Z0-9_]+) = convert\(\n {4}"""\n(.*?)"""\n\)', re.S)
file_template = '# file {filename}\n{variable} = convert(\n """\n{data}"""\n)'
def rebuild(script_path):
with open(script_path, "rt") as current_fh:
script_content = current_fh.read()
script_parts = []
match_end = 0
next_match = None
_count, did_update = 0, False
for _count, next_match in enumerate(file_regex.finditer(script_content)):
script_parts += [script_content[match_end : next_match.start()]]
match_end = next_match.end()
filename, variable_name, previous_encoded = next_match.group(1), next_match.group(2), next_match.group(3)
differ, content = handle_file(next_match.group(0), filename, variable_name, previous_encoded)
script_parts.append(content)
if differ:
did_update = True
script_parts += [script_content[match_end:]]
new_content = "".join(script_parts)
report(1 if not _count or did_update else 0, new_content, next_match, script_content, script_path)
def handle_file(previous_content, filename, variable_name, previous_encoded):
print("Found file {}".format(filename))
current_path = os.path.realpath(os.path.join(here, "..", "src", "virtualenv_embedded", filename))
_, file_type = os.path.splitext(current_path)
keep_line_ending = file_type in (".bat",)
with open(current_path, "rt", encoding="utf-8", newline="" if keep_line_ending else None) as current_fh:
current_text = current_fh.read()
current_crc = crc32(current_text)
current_encoded = b64.encode(gzip.encode(current_text.encode())[0])[0].decode()
if current_encoded == previous_encoded:
print(" File up to date (crc: {:08x})".format(current_crc))
return False, previous_content
# Else: content has changed
previous_text = gzip.decode(b64.decode(previous_encoded.encode())[0])[0].decode()
previous_crc = crc32(previous_text)
print(" Content changed (crc: {:08x} -> {:08x})".format(previous_crc, current_crc))
new_part = file_template.format(filename=filename, variable=variable_name, data=current_encoded)
return True, new_part
def report(exit_code, new, next_match, current, script_path):
if new != current:
print("Content updated; overwriting... ", end="")
with open(script_path, "wt") as current_fh:
current_fh.write(new)
print("done.")
else:
print("No changes in content")
if next_match is None:
print("No variables were matched/found")
raise SystemExit(exit_code)
if __name__ == "__main__":
rebuild(script)

107
tasks/upgrade_wheels.py Normal file
View file

@ -0,0 +1,107 @@
"""
Helper script to rebuild virtualenv_support. Downloads the wheel files using pip
"""
from __future__ import absolute_import, unicode_literals
import os
import shutil
import subprocess
import sys
from collections import OrderedDict, defaultdict
from tempfile import TemporaryDirectory
from threading import Thread
from pathlib2 import Path
STRICT = "UPGRADE_ADVISORY" not in os.environ
BUNDLED = ["pip", "setuptools"]
SUPPORT = list(reversed([(2, 7)] + [(3, i) for i in range(4, 10)]))
DEST = Path(__file__).resolve().parents[1] / "src" / "virtualenv" / "seed" / "wheels"
def download(ver, dest, package):
subprocess.call(
[sys.executable, "-m", "pip", "download", "--only-binary=:all:", "--python-version", ver, "-d", dest, package]
)
def run():
old_batch = {i.name for i in DEST.iterdir() if i.suffix == ".whl"}
with TemporaryDirectory() as temp:
temp_path = Path(temp)
folders = {}
targets = []
for support in SUPPORT:
support_ver = ".".join(str(i) for i in support)
into = temp_path / support_ver
into.mkdir()
folders[into] = support_ver
for package in BUNDLED:
thread = Thread(target=download, args=(support_ver, str(into), package))
targets.append(thread)
thread.start()
for thread in targets:
thread.join()
new_batch = {i.name: i for f in folders.keys() for i in Path(f).iterdir()}
new_packages = new_batch.keys() - old_batch
remove_packages = old_batch - new_batch.keys()
for package in remove_packages:
(DEST / package).unlink()
for package in new_packages:
shutil.copy2(str(new_batch[package]), DEST / package)
added = collect_package_versions(new_packages)
removed = collect_package_versions(remove_packages)
outcome = (1 if STRICT else 0) if (added or removed) else 0
for key, versions in added.items():
text = "* upgrade embedded {} to {}".format(key, fmt_version(versions))
if key in removed:
text += " from {}".format(removed[key])
del removed[key]
print(text)
for key, versions in removed.items():
print("* removed embedded {} of {}".format(key, fmt_version(versions)))
support_table = OrderedDict((".".join(str(j) for j in i), list()) for i in SUPPORT)
for package in sorted(new_batch.keys()):
for folder, version in sorted(folders.items()):
if (folder / package).exists():
support_table[version].append(package)
support_table = {k: OrderedDict((i.split("-")[0], i) for i in v) for k, v in support_table.items()}
msg = "from __future__ import absolute_import, unicode_literals; BUNDLE_SUPPORT = {{ {} }}; MAX = {!r}".format(
",".join(
"{!r}: {{ {} }}".format(v, ",".join("{!r}: {!r}".format(p, f) for p, f in l.items()))
for v, l in support_table.items()
),
next(iter(support_table.keys())),
)
dest_target = DEST / "__init__.py"
dest_target.write_text(msg)
subprocess.run([sys.executable, "-m", "black", str(dest_target)])
raise SystemExit(outcome)
def fmt_version(versions):
return ", ".join("``{}``".format(v) for v in versions)
def collect_package_versions(new_packages):
result = defaultdict(list)
for package in new_packages:
split = package.split("-")
if len(split) < 2:
raise ValueError(package)
key, version = split[0:2]
result[key].append(version)
return result
if __name__ == "__main__":
run()

209
tests/conftest.py Normal file
View file

@ -0,0 +1,209 @@
from __future__ import absolute_import, unicode_literals
import os
import shutil
import sys
from functools import partial
import coverage
import pytest
from pathlib2 import Path
from virtualenv.interpreters.discovery.py_info import CURRENT
@pytest.fixture(scope="session")
def has_symlink_support(tmp_path_factory):
platform_supports = hasattr(os, "symlink")
if platform_supports and sys.platform == "win32":
# on Windows correct functioning of this is tied to SeCreateSymbolicLinkPrivilege, try if it works
test_folder = tmp_path_factory.mktemp("symlink-tests")
src = test_folder / "src"
try:
src.symlink_to(test_folder / "dest")
except OSError:
return False
finally:
shutil.rmtree(str(test_folder))
return platform_supports
@pytest.fixture(scope="session")
def link_folder(has_symlink_support):
if has_symlink_support:
return os.symlink
elif sys.platform == "win32" and sys.version_info[0:2] > (3, 4):
# on Windows junctions may be used instead
import _winapi # python3.5 has builtin implementation for junctions
return _winapi.CreateJunction
else:
return None
@pytest.fixture(scope="session")
def link_file(has_symlink_support):
if has_symlink_support:
return os.symlink
else:
return None
@pytest.fixture(scope="session")
def link(link_folder, link_file):
def _link(src, dest):
clean = dest.unlink
s_dest = str(dest)
s_src = str(src)
if src.is_dir():
if link_folder:
link_folder(s_src, s_dest)
else:
shutil.copytree(s_src, s_dest)
clean = partial(shutil.rmtree, str(dest))
else:
if link_file:
link_file(s_src, s_dest)
else:
shutil.copy2(s_src, s_dest)
return clean
return _link
@pytest.fixture(autouse=True)
def check_cwd_not_changed_by_test():
old = os.getcwd()
yield
new = os.getcwd()
if old != new:
pytest.fail("tests changed cwd: {!r} => {!r}".format(old, new))
@pytest.fixture(autouse=True)
def clean_data_dir(tmp_path, monkeypatch):
from virtualenv import info
monkeypatch.setattr(info, "_DATA_DIR", tmp_path)
yield
@pytest.fixture(autouse=True)
def check_os_environ_stable():
old = os.environ.copy()
# ensure we don't inherit parent env variables
to_clean = {
k for k in os.environ.keys() if k.startswith("VIRTUALENV_") or "VIRTUAL_ENV" in k or k.startswith("TOX_")
}
cleaned = {k: os.environ[k] for k, v in os.environ.items()}
os.environ[str("VIRTUALENV_NO_DOWNLOAD")] = str("1")
is_exception = False
try:
yield
except BaseException:
is_exception = True
raise
finally:
try:
del os.environ[str("VIRTUALENV_NO_DOWNLOAD")]
if is_exception is False:
new = os.environ
extra = {k: new[k] for k in set(new) - set(old)}
miss = {k: old[k] for k in set(old) - set(new) - to_clean}
diff = {
"{} = {} vs {}".format(k, old[k], new[k])
for k in set(old) & set(new)
if old[k] != new[k] and not k.startswith("PYTEST_")
}
if extra or miss or diff:
msg = "tests changed environ"
if extra:
msg += " extra {}".format(extra)
if miss:
msg += " miss {}".format(miss)
if diff:
msg += " diff {}".format(diff)
pytest.fail(msg)
finally:
os.environ.update(cleaned)
COV_ENV_VAR = "COVERAGE_PROCESS_START"
COVERAGE_RUN = os.environ.get(COV_ENV_VAR)
@pytest.fixture(autouse=True)
def coverage_env(monkeypatch, link):
"""
Enable coverage report collection on the created virtual environments by injecting the coverage project
"""
if COVERAGE_RUN:
# we inject right after creation, we cannot collect coverage on site.py - used for helper scripts, such as debug
from virtualenv import run
def via_cli(args):
session = prev_run(args)
old_run = session.creator.run
def create_run():
result = old_run()
obj["cov"] = EnableCoverage(link)
obj["cov"].__enter__(session.creator)
return result
monkeypatch.setattr(session.creator, "run", create_run)
return session
obj = {"cov": None}
prev_run = run.session_via_cli
monkeypatch.setattr(run, "session_via_cli", via_cli)
def finish():
cov = obj["cov"]
obj["cov"] = None
cov.__exit__(None, None, None)
yield finish
if obj["cov"]:
finish()
else:
def finish():
pass
yield finish
class EnableCoverage(object):
_COV = Path(coverage.__file__).parents[1]
ENTRIES = [i for i in _COV.iterdir() if i.name.startswith("coverage")]
_COV_DEVICE = _COV.stat().st_dev
def __init__(self, link):
self.link = link
self.targets = []
self._entered = False
def __enter__(self, creator):
self._entered = True
site_packages = creator.site_packages[0]
if str(site_packages) not in CURRENT.path:
for entry in self.ENTRIES:
target = site_packages / entry.name
if not target.exists():
clean = self.link(entry, target)
self.targets.append((target, clean))
p_th = site_packages / "coverage-virtualenv.pth"
if str(p_th.resolve()).startswith(r"C:\Users\traveler\git\virtualenv\.tox"):
raise ValueError(site_packages)
p_th.write_text("import coverage; coverage.process_startup()")
self.targets.append((p_th, p_th.unlink))
return self
def __exit__(self, exc_type, exc_val, exc_tb):
if self._entered:
for target, clean in self.targets:
if target.exists():
clean()

38
tests/unit/config/cli/test_parser.py Normal file
View file

@ -0,0 +1,38 @@
from __future__ import absolute_import, unicode_literals
import os
from contextlib import contextmanager
import pytest
from virtualenv.config.cli.parser import VirtualEnvConfigParser
from virtualenv.config.ini import IniConfig
@pytest.fixture()
def gen_parser_no_conf_env(monkeypatch, tmp_path):
keys_to_delete = {key for key in os.environ if key.startswith("VIRTUALENV_")}
for key in keys_to_delete:
monkeypatch.delenv(key)
monkeypatch.setenv(IniConfig.VIRTUALENV_CONFIG_FILE_ENV_VAR, str(tmp_path / "missing"))
@contextmanager
def _build():
parser = VirtualEnvConfigParser()
def _run(*args):
return parser.parse_args(args=args)
yield parser, _run
parser.enable_help()
return _build
def test_flag(gen_parser_no_conf_env):
with gen_parser_no_conf_env() as (parser, run):
parser.add_argument("--clear", dest="clear", action="store_true", help="it", default=False)
result = run()
assert result.clear is False
result = run("--clear")
assert result.clear is True

9
tests/unit/config/test___main__.py Normal file
View file

@ -0,0 +1,9 @@
from __future__ import absolute_import, unicode_literals
import subprocess
import sys
def test_main():
out = subprocess.check_output([sys.executable, "-m", "virtualenv", "--help"], universal_newlines=True)
assert out

39
tests/unit/config/test_env_var.py Normal file
View file

@ -0,0 +1,39 @@
from __future__ import absolute_import, unicode_literals
import pytest
from virtualenv.config.ini import IniConfig
from virtualenv.run import session_via_cli
def parse_cli(args):
return session_via_cli(args)
@pytest.fixture()
def empty_conf(tmp_path, monkeypatch):
conf = tmp_path / "conf.ini"
monkeypatch.setenv(IniConfig.VIRTUALENV_CONFIG_FILE_ENV_VAR, str(conf))
conf.write_text("[virtualenv]")
def test_value_ok(monkeypatch, empty_conf):
monkeypatch.setenv(str("VIRTUALENV_VERBOSE"), str("5"))
result = parse_cli([])
assert result.verbosity == 5
def _exc(of):
try:
int(of)
except ValueError as exception:
return exception
def test_value_bad(monkeypatch, caplog, empty_conf):
monkeypatch.setenv(str("VIRTUALENV_VERBOSE"), str("a"))
result = parse_cli([])
assert result.verbosity == 3
msg = "env var VIRTUALENV_VERBOSE failed to convert 'a' as {!r} because {!r}".format(int, _exc("a"))
# one for the core parse, one for the normal one
assert caplog.messages == [msg], "{}{}".format(caplog.text, msg)

18
tests/unit/interpreters/boostrap/perf.py Normal file
View file

@ -0,0 +1,18 @@
from __future__ import absolute_import, unicode_literals
from virtualenv.interpreters.discovery import CURRENT
from virtualenv.run import run_via_cli
from virtualenv.seed.wheels import BUNDLE_SUPPORT
dest = r"C:\Users\traveler\git\virtualenv\test\unit\interpreters\boostrap\perf"
bundle_ver = BUNDLE_SUPPORT[CURRENT.version_release_str]
cmd = [
dest,
"--download",
"--pip",
bundle_ver["pip"].split("-")[1],
"--setuptools",
bundle_ver["setuptools"].split("-")[1],
]
result = run_via_cli(cmd)
assert result

78
tests/unit/interpreters/boostrap/test_boostrap_link_via_app_data.py Normal file
View file

@ -0,0 +1,78 @@
from __future__ import absolute_import, unicode_literals
import os
import subprocess
import sys
from virtualenv.interpreters.discovery.py_info import CURRENT
from virtualenv.run import run_via_cli
from virtualenv.seed.embed.wheels import BUNDLE_SUPPORT
def test_base_bootstrap_link_via_app_data(tmp_path, coverage_env):
bundle_ver = BUNDLE_SUPPORT[CURRENT.version_release_str]
create_cmd = [
str(tmp_path / "env"),
"--download",
"--pip",
bundle_ver["pip"].split("-")[1],
"--setuptools",
bundle_ver["setuptools"].split("-")[1],
]
result = run_via_cli(create_cmd)
coverage_env()
assert result
# uninstalling pip/setuptools now should leave us with a clean env
site_package = result.creator.site_packages[0]
pip = site_package / "pip"
setuptools = site_package / "setuptools"
files_post_first_create = list(site_package.iterdir())
assert pip in files_post_first_create
assert setuptools in files_post_first_create
env_exe = result.creator.env_exe
for pip_exe in [
env_exe.with_name("pip{}{}".format(suffix, env_exe.suffix))
for suffix in (
"",
"{}".format(CURRENT.version_info.major),
"-{}.{}".format(CURRENT.version_info.major, CURRENT.version_info.minor),
)
]:
assert pip_exe.exists()
subprocess.check_output([str(pip_exe), "--version", "--disable-pip-version-check"])
remove_cmd = [
str(env_exe),
"-m",
"pip",
"--verbose",
"--disable-pip-version-check",
"uninstall",
"-y",
"setuptools",
]
assert not subprocess.check_call(remove_cmd)
assert site_package.exists()
files_post_first_uninstall = list(site_package.iterdir())
assert pip in files_post_first_uninstall
assert setuptools not in files_post_first_uninstall
# check we can run it again and will work - checks both overwrite and reuse cache
result = run_via_cli(create_cmd)
coverage_env()
assert result
files_post_second_create = list(site_package.iterdir())
assert files_post_first_create == files_post_second_create
assert not subprocess.check_call(remove_cmd + ["pip"])
# pip is greedy here, removing all packages removes the site-package too
if site_package.exists():
post_run = list(site_package.iterdir())
assert not post_run, "\n".join(str(i) for i in post_run)
if sys.version_info[0:2] == (3, 4) and "PIP_REQ_TRACKER" in os.environ:
os.environ.pop("PIP_REQ_TRACKER")

62
tests/unit/interpreters/create/conftest.py Normal file
View file

@ -0,0 +1,62 @@
"""
It's possible to use multiple types of host pythons to create virtual environments and all should work:
- host installation
- invoking from a venv (if Python 3.3+)
- invoking from an old style virtualenv (<17.0.0)
- invoking from our own venv
"""
from __future__ import absolute_import, unicode_literals
import subprocess
import sys
import pytest
from virtualenv.interpreters.discovery.py_info import CURRENT, IS_WIN
# noinspection PyUnusedLocal
def get_root(tmp_path_factory):
return CURRENT.system_executable
def get_venv(tmp_path_factory):
if CURRENT.is_venv:
return sys.executable
elif CURRENT.version_info.major == 3:
root_python = get_root(tmp_path_factory)
dest = tmp_path_factory.mktemp("venv")
subprocess.check_call([str(root_python), "-m", "venv", "--without-pip", str(dest)])
return CURRENT.find_exe(str(dest))
def get_virtualenv(tmp_path_factory):
if CURRENT.is_old_virtualenv:
return CURRENT.executable
elif CURRENT.version_info.major == 3:
# noinspection PyCompatibility
from venv import EnvBuilder
virtualenv_at = str(tmp_path_factory.mktemp("venv-for-virtualenv"))
builder = EnvBuilder(symlinks=not IS_WIN)
builder.create(virtualenv_at)
venv_for_virtualenv = CURRENT.find_exe(virtualenv_at)
cmd = venv_for_virtualenv, "-m", "pip", "install", "virtualenv==16.6.1"
subprocess.check_call(cmd)
virtualenv_python = tmp_path_factory.mktemp("virtualenv")
cmd = venv_for_virtualenv, "-m", "virtualenv", virtualenv_python
subprocess.check_call(cmd)
return CURRENT.find_exe(virtualenv_python)
PYTHON = {"root": get_root, "venv": get_venv, "virtualenv": get_virtualenv}
@pytest.fixture(params=list(PYTHON.values()), ids=list(PYTHON.keys()), scope="session")
def python(request, tmp_path_factory):
result = request.param(tmp_path_factory)
if result is None:
pytest.skip("could not resolve {}".format(request.param))
return result

204
tests/unit/interpreters/create/test_creator.py Normal file
View file

@ -0,0 +1,204 @@
from __future__ import absolute_import, unicode_literals
import difflib
import os
import stat
import sys
import pytest
import six
from pathlib2 import Path
from virtualenv.__main__ import run
from virtualenv.interpreters.create.creator import DEBUG_SCRIPT, get_env_debug_info
from virtualenv.interpreters.discovery.py_info import CURRENT
from virtualenv.pyenv_cfg import PyEnvCfg
from virtualenv.run import run_via_cli, session_via_cli
def test_os_path_sep_not_allowed(tmp_path, capsys):
target = str(tmp_path / "a{}b".format(os.pathsep))
err = _non_success_exit_code(capsys, target)
msg = (
"destination {!r} must not contain the path separator ({}) as this"
" would break the activation scripts".format(target, os.pathsep)
)
assert msg in err, err
def _non_success_exit_code(capsys, target):
with pytest.raises(SystemExit) as context:
run_via_cli(args=[target])
assert context.value.code != 0
out, err = capsys.readouterr()
assert not out, out
return err
def test_destination_exists_file(tmp_path, capsys):
target = tmp_path / "out"
target.write_text("")
err = _non_success_exit_code(capsys, str(target))
msg = "the destination {} already exists and is a file".format(str(target))
assert msg in err, err
@pytest.mark.skipif(sys.platform == "win32", reason="no chmod on Windows")
def test_destination_not_write_able(tmp_path, capsys):
target = tmp_path
prev_mod = target.stat().st_mode
target.chmod(0o444)
try:
err = _non_success_exit_code(capsys, str(target))
msg = "the destination . is not write-able at {}".format(str(target))
assert msg in err, err
finally:
target.chmod(prev_mod)
SYSTEM = get_env_debug_info(CURRENT.system_executable, DEBUG_SCRIPT)
def cleanup_sys_path(path):
from virtualenv.interpreters.create.creator import HERE
path = [Path(i).absolute() for i in path]
to_remove = [Path(HERE)]
if str("PYCHARM_HELPERS_DIR") in os.environ:
to_remove.append(Path(os.environ[str("PYCHARM_HELPERS_DIR")]).parent / "pydev")
for elem in to_remove:
try:
index = path.index(elem)
del path[index]
except ValueError:
pass
return path
@pytest.mark.parametrize("global_access", [False, True], ids=["no_global", "ok_global"])
@pytest.mark.parametrize(
"use_venv", [False, True] if six.PY3 else [False], ids=["no_venv", "venv"] if six.PY3 else ["no_venv"]
)
def test_create_no_seed(python, use_venv, global_access, tmp_path, coverage_env):
cmd = ["-v", "-v", "-p", str(python), str(tmp_path), "--without-pip"]
if global_access:
cmd.append("--system-site-packages")
if use_venv:
cmd.extend(["--creator", "venv"])
result = run_via_cli(cmd)
coverage_env()
for site_package in result.creator.site_packages:
content = list(site_package.iterdir())
assert not content, "\n".join(str(i) for i in content)
assert result.creator.env_name == tmp_path.name
sys_path = cleanup_sys_path(result.creator.debug["sys"]["path"])
system_sys_path = cleanup_sys_path(SYSTEM["sys"]["path"])
our_paths = set(sys_path) - set(system_sys_path)
our_paths_repr = "\n".join(repr(i) for i in our_paths)
# ensure we have at least one extra path added
assert len(our_paths) >= 1, our_paths_repr
# ensure all additional paths are related to the virtual environment
for path in our_paths:
assert str(path).startswith(str(tmp_path)), path
# ensure there's at least a site-packages folder as part of the virtual environment added
assert any(p for p in our_paths if p.parts[-1] == "site-packages"), our_paths_repr
# ensure the global site package is added or not, depending on flag
last_from_system_path = next(i for i in reversed(system_sys_path) if str(i).startswith(SYSTEM["sys"]["prefix"]))
if global_access:
common = []
for left, right in zip(reversed(system_sys_path), reversed(sys_path)):
if left == right:
common.append(left)
else:
break
def list_to_str(iterable):
return [str(i) for i in iterable]
assert common, "\n".join(difflib.unified_diff(list_to_str(sys_path), list_to_str(system_sys_path)))
else:
assert last_from_system_path not in sys_path
@pytest.mark.skipif(not CURRENT.has_venv, reason="requires venv interpreter")
def test_venv_fails_not_inline(tmp_path, capsys, mocker):
def _session_via_cli(args):
session = session_via_cli(args)
assert session.creator.can_be_inline is False
return session
mocker.patch("virtualenv.run.session_via_cli", side_effect=_session_via_cli)
before = tmp_path.stat().st_mode
cfg_path = tmp_path / "pyvenv.cfg"
cfg_path.write_text(six.ensure_text(""))
cfg = str(cfg_path)
try:
os.chmod(cfg, stat.S_IREAD | stat.S_IRGRP | stat.S_IROTH)
cmd = ["-p", str(CURRENT.executable), str(tmp_path), "--without-pip", "--creator", "venv"]
with pytest.raises(SystemExit) as context:
run(cmd)
assert context.value.code != 0
finally:
os.chmod(cfg, before)
out, err = capsys.readouterr()
assert "subprocess call failed for" in out, out
assert "Error:" in err, err
@pytest.mark.skipif(not sys.version_info[0] == 2, reason="python 2 only tests")
def test_debug_bad_virtualenv(tmp_path):
cmd = [str(tmp_path), "--without-pip"]
result = run_via_cli(cmd)
# if the site.py is removed/altered the debug should fail as no one is around to fix the paths
site_py = result.creator.lib_dir / "site.py"
site_py.unlink()
# insert something that writes something on the stdout
site_py.write_text('import sys; sys.stdout.write(repr("std-out")); sys.stderr.write("std-err"); raise ValueError')
debug_info = result.creator.debug
assert debug_info["returncode"]
assert debug_info["err"].startswith("std-err")
assert debug_info["out"] == "'std-out'"
assert debug_info["exception"]
@pytest.mark.parametrize(
"use_venv", [False, True] if six.PY3 else [False], ids=["no_venv", "venv"] if six.PY3 else ["no_venv"]
)
@pytest.mark.parametrize("clear", [True, False], ids=["clear", "no_clear"])
def test_create_clear_resets(tmp_path, use_venv, clear):
marker = tmp_path / "magic"
cmd = [str(tmp_path), "--without-pip"]
if use_venv:
cmd.extend(["--creator", "venv"])
run_via_cli(cmd)
marker.write_text("") # if we a marker file this should be gone on a clear run, remain otherwise
assert marker.exists()
run_via_cli(cmd + (["--clear"] if clear else []))
assert marker.exists() is not clear
@pytest.mark.skip
@pytest.mark.parametrize(
"use_venv", [False, True] if six.PY3 else [False], ids=["no_venv", "venv"] if six.PY3 else ["no_venv"]
)
@pytest.mark.parametrize("prompt", [None, "magic"])
def test_prompt_set(tmp_path, use_venv, prompt):
cmd = [str(tmp_path), "--without-pip"]
if prompt is not None:
cmd.extend(["--prompt", "magic"])
if not use_venv:
cmd.extend(["--creator", "venv"])
result = run_via_cli(cmd)
actual_prompt = tmp_path.name if prompt is None else prompt
cfg = PyEnvCfg.from_file(result.creator.pyenv_cfg.path)
if prompt is None:
assert "prompt" not in cfg
else:
if use_venv is False:
assert "prompt" in cfg, list(cfg.content.keys())
assert cfg["prompt"] == actual_prompt

34
tests/unit/interpreters/discovery/test_discovery.py Normal file
View file

@ -0,0 +1,34 @@
from __future__ import absolute_import, unicode_literals
import os
import sys
from uuid import uuid4
import pytest
from virtualenv.interpreters.discovery.builtin import get_interpreter
from virtualenv.interpreters.discovery.py_info import CURRENT
@pytest.mark.skipif(sys.platform == "win32", reason="symlink is not guaranteed to work on windows")
@pytest.mark.parametrize("lower", [None, True, False])
def test_discovery_via_path(tmp_path, monkeypatch, lower):
core = "somethingVeryCryptic{}".format(".".join(str(i) for i in CURRENT.version_info[0:3]))
name = "somethingVeryCryptic"
if lower is True:
name = name.lower()
elif lower is False:
name = name.upper()
exe_name = "{}{}{}".format(name, CURRENT.version_info.major, ".exe" if sys.platform == "win32" else "")
executable = tmp_path / exe_name
os.symlink(sys.executable, str(executable))
new_path = os.pathsep.join([str(tmp_path)] + os.environ.get(str("PATH"), str("")).split(os.pathsep))
monkeypatch.setenv(str("PATH"), new_path)
interpreter = get_interpreter(core)
assert interpreter is not None
def test_discovery_via_path_not_found():
interpreter = get_interpreter(uuid4().hex)
assert interpreter is None

90
tests/unit/interpreters/discovery/test_py_info.py Normal file
View file

@ -0,0 +1,90 @@
from __future__ import absolute_import, unicode_literals
import itertools
import json
import logging
import sys
import pytest
from virtualenv.interpreters.discovery.py_info import CURRENT, PythonInfo
from virtualenv.interpreters.discovery.py_spec import PythonSpec
def test_current_as_json():
result = CURRENT.to_json()
parsed = json.loads(result)
a, b, c, d, e = sys.version_info
assert parsed["version_info"] == {"major": a, "minor": b, "micro": c, "releaselevel": d, "serial": e}
def test_bad_exe_py_info_raise(tmp_path):
exe = str(tmp_path)
with pytest.raises(RuntimeError) as context:
PythonInfo.from_exe(exe)
msg = str(context.value)
assert "code" in msg
assert exe in msg
def test_bad_exe_py_info_no_raise(tmp_path, caplog, capsys):
caplog.set_level(logging.NOTSET)
exe = str(tmp_path)
result = PythonInfo.from_exe(exe, raise_on_error=False)
assert result is None
out, _ = capsys.readouterr()
assert not out
assert len(caplog.messages) == 1
msg = caplog.messages[0]
assert repr(exe) in msg
assert "code" in msg
@pytest.mark.parametrize(
"spec",
itertools.chain(
[sys.executable],
list(
"{}{}{}".format(impl, ".".join(str(i) for i in ver), arch)
for impl, ver, arch in itertools.product(
([CURRENT.implementation] + (["python"] if CURRENT.implementation == "CPython" else [])),
[sys.version_info[0 : i + 1] for i in range(3)],
["", "-{}".format(CURRENT.architecture)],
)
),
),
)
def test_satisfy_py_info(spec):
parsed_spec = PythonSpec.from_string_spec(spec)
matches = CURRENT.satisfies(parsed_spec, True)
assert matches is True
def test_satisfy_not_arch():
parsed_spec = PythonSpec.from_string_spec(
"{}-{}".format(CURRENT.implementation, 64 if CURRENT.architecture == 32 else 32)
)
matches = CURRENT.satisfies(parsed_spec, True)
assert matches is False
def _generate_not_match_current_interpreter_version():
result = []
for i in range(3):
ver = sys.version_info[0 : i + 1]
for a in range(len(ver)):
for o in [-1, 1]:
temp = list(ver)
temp[a] += o
result.append(".".join(str(i) for i in temp))
return result
_NON_MATCH_VER = _generate_not_match_current_interpreter_version()
@pytest.mark.parametrize("spec", _NON_MATCH_VER)
def test_satisfy_not_version(spec):
parsed_spec = PythonSpec.from_string_spec("{}{}".format(CURRENT.implementation, spec))
matches = CURRENT.satisfies(parsed_spec, True)
assert matches is False

106
tests/unit/interpreters/discovery/test_py_spec.py Normal file
View file

@ -0,0 +1,106 @@
from __future__ import absolute_import, unicode_literals
import itertools
import sys
from copy import copy
import pytest
from virtualenv.interpreters.discovery.py_spec import PythonSpec
def test_bad_py_spec():
text = "python2.3.4.5"
spec = PythonSpec.from_string_spec(text)
assert text in repr(spec)
assert spec.str_spec == text
assert spec.path == text
content = vars(spec)
del content[str("str_spec")]
del content[str("path")]
assert all(v is None for v in content.values())
def test_py_spec_first_digit_only_major():
spec = PythonSpec.from_string_spec("python278")
assert spec.major == 2
assert spec.minor == 78
def test_spec_satisfies_path_ok():
spec = PythonSpec.from_string_spec(sys.executable)
assert spec.satisfies(spec) is True
def test_spec_satisfies_path_nok(tmp_path):
spec = PythonSpec.from_string_spec(sys.executable)
of = PythonSpec.from_string_spec(str(tmp_path))
assert spec.satisfies(of) is False
def test_spec_satisfies_arch():
spec_1 = PythonSpec.from_string_spec("python-32")
spec_2 = PythonSpec.from_string_spec("python-64")
assert spec_1.satisfies(spec_1) is True
assert spec_2.satisfies(spec_1) is False
@pytest.mark.parametrize(
"req, spec", list(itertools.combinations(["py", "CPython", "python"], 2)) + [("jython", "jython")]
)
def test_spec_satisfies_implementation_ok(req, spec):
spec_1 = PythonSpec.from_string_spec(req)
spec_2 = PythonSpec.from_string_spec(spec)
assert spec_1.satisfies(spec_1) is True
assert spec_2.satisfies(spec_1) is True
def test_spec_satisfies_implementation_nok():
spec_1 = PythonSpec.from_string_spec("python")
spec_2 = PythonSpec.from_string_spec("jython")
assert spec_2.satisfies(spec_1) is False
assert spec_1.satisfies(spec_2) is False
def _version_satisfies_pairs():
target = set()
version = tuple(str(i) for i in sys.version_info[0:3])
for i in range(len(version) + 1):
req = ".".join(version[0:i])
for j in range(i + 1):
sat = ".".join(version[0:j])
# can be satisfied in both directions
target.add((req, sat))
target.add((sat, req))
return sorted(target)
@pytest.mark.parametrize("req, spec", _version_satisfies_pairs())
def test_version_satisfies_ok(req, spec):
req_spec = PythonSpec.from_string_spec("python{}".format(req))
sat_spec = PythonSpec.from_string_spec("python{}".format(spec))
assert sat_spec.satisfies(req_spec) is True
def _version_not_satisfies_pairs():
target = set()
version = tuple(str(i) for i in sys.version_info[0:3])
for i in range(len(version)):
req = ".".join(version[0 : i + 1])
for j in range(i + 1):
sat_ver = list(sys.version_info[0 : j + 1])
for l in range(j + 1):
for o in [1, -1]:
temp = copy(sat_ver)
temp[l] += o
sat = ".".join(str(i) for i in temp)
target.add((req, sat))
return sorted(target)
@pytest.mark.parametrize("req, spec", _version_not_satisfies_pairs())
def test_version_satisfies_nok(req, spec):
req_spec = PythonSpec.from_string_spec("python{}".format(req))
sat_spec = PythonSpec.from_string_spec("python{}".format(spec))
assert sat_spec.satisfies(req_spec) is False

195
tests/unit/interpreters/discovery/windows/test_windows_pep514.py Normal file
View file

@ -0,0 +1,195 @@
from __future__ import absolute_import, unicode_literals
import sys
import textwrap
from collections import defaultdict
from contextlib import contextmanager
import pytest
import six
from pathlib2 import Path
@pytest.mark.skipif(sys.platform != "win32", reason="Windows registry only on Windows platform")
def test_pep517(_mock_registry):
from virtualenv.interpreters.discovery.windows.pep514 import discover_pythons
interpreters = list(discover_pythons())
assert interpreters == [
("ContinuumAnalytics", 3, 7, 32, "C:\\Users\\traveler\\Miniconda3\\python.exe", None),
("ContinuumAnalytics", 3, 7, 64, "C:\\Users\\traveler\\Miniconda3-64\\python.exe", None),
("python", 3, 6, 64, "C:\\Users\\traveler\\AppData\\Local\\Programs\\Python\\Python36\\python.exe", None),
("python", 3, 6, 64, "C:\\Users\\traveler\\AppData\\Local\\Programs\\Python\\Python36\\python.exe", None),
("python", 3, 5, 64, "C:\\Users\\traveler\\AppData\\Local\\Programs\\Python\\Python35\\python.exe", None),
("python", 3, 6, 64, "C:\\Users\\traveler\\AppData\\Local\\Programs\\Python\\Python36\\python.exe", None),
("python", 3, 7, 32, "C:\\Users\\traveler\\AppData\\Local\\Programs\\Python\\Python37-32\\python.exe", None),
("python", 3, 9, 64, "C:\\Users\\traveler\\AppData\\Local\\Programs\\Python\\Python36\\python.exe", None),
("python", 2, 7, 64, "C:\\Python27\\python.exe", None),
("python", 3, 4, 64, "C:\\Python34\\python.exe", None),
]
@pytest.mark.skipif(sys.platform != "win32", reason="Windows registry only on Windows platform")
def test_pep517_run(_mock_registry, capsys, caplog):
from virtualenv.interpreters.discovery.windows import pep514
pep514._run()
out, err = capsys.readouterr()
expected = textwrap.dedent(
r"""
('ContinuumAnalytics', 3, 7, 32, 'C:\\Users\\traveler\\Miniconda3\\python.exe', None)
('ContinuumAnalytics', 3, 7, 64, 'C:\\Users\\traveler\\Miniconda3-64\\python.exe', None)
('python', 2, 7, 64, 'C:\\Python27\\python.exe', None)
('python', 3, 4, 64, 'C:\\Python34\\python.exe', None)
('python', 3, 5, 64, 'C:\\Users\\traveler\\AppData\\Local\\Programs\\Python\\Python35\\python.exe', None)
('python', 3, 6, 64, 'C:\\Users\\traveler\\AppData\\Local\\Programs\\Python\\Python36\\python.exe', None)
('python', 3, 6, 64, 'C:\\Users\\traveler\\AppData\\Local\\Programs\\Python\\Python36\\python.exe', None)
('python', 3, 6, 64, 'C:\\Users\\traveler\\AppData\\Local\\Programs\\Python\\Python36\\python.exe', None)
('python', 3, 7, 32, 'C:\\Users\\traveler\\AppData\\Local\\Programs\\Python\\Python37-32\\python.exe', None)
('python', 3, 9, 64, 'C:\\Users\\traveler\\AppData\\Local\\Programs\\Python\\Python36\\python.exe', None)
"""
).strip()
assert out.strip() == expected
assert not err
prefix = "PEP-514 violation in Windows Registry at "
expected_logs = [
"{}HKEY_CURRENT_USER/PythonCore/3.1/SysArchitecture error: invalid format magic".format(prefix),
"{}HKEY_CURRENT_USER/PythonCore/3.2/SysArchitecture error: arch is not string: 100".format(prefix),
"{}HKEY_CURRENT_USER/PythonCore/3.3 error: no ExecutablePath or default for it".format(prefix),
"{}HKEY_CURRENT_USER/PythonCore/3.3 error: exe does not exists HKEY_CURRENT_USER/PythonCore/3.3".format(prefix),
"{}HKEY_CURRENT_USER/PythonCore/3.8/InstallPath error: missing".format(prefix),
"{}HKEY_CURRENT_USER/PythonCore/3.9/SysVersion error: invalid format magic".format(prefix),
"{}HKEY_CURRENT_USER/PythonCore/3.X/SysVersion error: version is not string: 2778".format(prefix),
"{}HKEY_CURRENT_USER/PythonCore/3.X error: invalid format 3.X".format(prefix),
]
assert caplog.messages == expected_logs
@pytest.fixture()
def _mock_registry(mocker):
from virtualenv.interpreters.discovery.windows.pep514 import winreg
loc, glob = {}, {}
mock_value_str = (Path(__file__).parent / "winreg-mock-values.py").read_text()
six.exec_(mock_value_str, glob, loc)
enum_collect = loc["enum_collect"]
value_collect = loc["value_collect"]
key_open = loc["key_open"]
hive_open = loc["hive_open"]
def _e(key, at):
key_id = key.value if isinstance(key, Key) else key
result = enum_collect[key_id][at]
if isinstance(result, OSError):
raise result
return result
mocker.patch.object(winreg, "EnumKey", side_effect=_e)
def _v(key, value_name):
key_id = key.value if isinstance(key, Key) else key
result = value_collect[key_id][value_name]
if isinstance(result, OSError):
raise result
return result
mocker.patch.object(winreg, "QueryValueEx", side_effect=_v)
class Key(object):
def __init__(self, value):
self.value = value
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb):
return None
@contextmanager
def _o(*args):
if len(args) == 2:
key, value = args
key_id = key.value if isinstance(key, Key) else key
result = Key(key_open[key_id][value]) # this needs to be something that can be with-ed, so let's wrap it
elif len(args) == 4:
result = hive_open[args]
else:
raise RuntimeError
value = result.value if isinstance(result, Key) else result
if isinstance(value, OSError):
raise value
yield result
mocker.patch.object(winreg, "OpenKeyEx", side_effect=_o)
mocker.patch("os.path.exists", return_value=True)
@pytest.fixture()
def _collect_winreg_access(mocker):
if six.PY3:
# noinspection PyUnresolvedReferences
from winreg import EnumKey, OpenKeyEx, QueryValueEx
else:
# noinspection PyUnresolvedReferences
from _winreg import EnumKey, OpenKeyEx, QueryValueEx
from virtualenv.interpreters.discovery.windows.pep514 import winreg
hive_open = {}
key_open = defaultdict(dict)
@contextmanager
def _c(*args):
res = None
key_id = id(args[0]) if len(args) == 2 else None
try:
with OpenKeyEx(*args) as c:
res = id(c)
yield c
except Exception as exception:
res = exception
raise exception
finally:
if len(args) == 4:
hive_open[args] = res
elif len(args) == 2:
key_open[key_id][args[1]] = res
enum_collect = defaultdict(list)
def _e(key, at):
result = None
key_id = id(key)
try:
result = EnumKey(key, at)
return result
except Exception as exception:
result = exception
raise result
finally:
enum_collect[key_id].append(result)
value_collect = defaultdict(dict)
def _v(key, value_name):
result = None
key_id = id(key)
try:
result = QueryValueEx(key, value_name)
return result
except Exception as exception:
result = exception
raise result
finally:
value_collect[key_id][value_name] = result
mocker.patch.object(winreg, "EnumKey", side_effect=_e)
mocker.patch.object(winreg, "QueryValueEx", side_effect=_v)
mocker.patch.object(winreg, "OpenKeyEx", side_effect=_c)
yield
print("")
print("hive_open = {}".format(hive_open))
print("key_open = {}".format(dict(key_open.items())))
print("value_collect = {}".format(dict(value_collect.items())))
print("enum_collect = {}".format(dict(enum_collect.items())))

136
tests/unit/interpreters/discovery/windows/winreg-mock-values.py Normal file
View file

@ -0,0 +1,136 @@
import six
if six.PY3:
import winreg
else:
# noinspection PyUnresolvedReferences
import _winreg as winreg
hive_open = {
(winreg.HKEY_CURRENT_USER, "Software\\Python", 0, winreg.KEY_READ): 78701856,
(winreg.HKEY_LOCAL_MACHINE, "Software\\Python", 0, winreg.KEY_READ | winreg.KEY_WOW64_64KEY): 78701840,
(winreg.HKEY_LOCAL_MACHINE, "Software\\Python", 0, winreg.KEY_READ | winreg.KEY_WOW64_32KEY): OSError(
2, "The system cannot find the file specified"
),
}
key_open = {
78701152: {
"Anaconda37-32\\InstallPath": 78703200,
"Anaconda37-32": 78703568,
"Anaconda37-64\\InstallPath": 78703520,
"Anaconda37-64": 78702368,
},
78701856: {"ContinuumAnalytics": 78701152, "PythonCore": 78702656},
78702656: {
"3.1\\InstallPath": 78701824,
"3.1": 78700704,
"3.2\\InstallPath": 78704048,
"3.2": 78704368,
"3.3\\InstallPath": 78701936,
"3.3": 78703024,
"3.5\\InstallPath": 78703792,
"3.5": 78701792,
"3.6\\InstallPath": 78701888,
"3.6": 78703424,
"3.7-32\\InstallPath": 78703600,
"3.7-32": 78704512,
"3.8\\InstallPath": OSError(2, "The system cannot find the file specified"),
"3.8": 78700656,
"3.9\\InstallPath": 78703632,
"3.9": 78702608,
"3.X": 78703088,
},
78702960: {"2.7\\InstallPath": 78700912, "2.7": 78703136, "3.4\\InstallPath": 78703648, "3.4": 78704032},
78701840: {"PythonCore": 78702960},
}
value_collect = {
78703568: {"SysVersion": ("3.7", 1), "SysArchitecture": ("32bit", 1)},
78703200: {
"ExecutablePath": ("C:\\Users\\traveler\\Miniconda3\\python.exe", 1),
"ExecutableArguments": OSError(2, "The system cannot find the file specified"),
},
78702368: {"SysVersion": ("3.7", 1), "SysArchitecture": ("64bit", 1)},
78703520: {
"ExecutablePath": ("C:\\Users\\traveler\\Miniconda3-64\\python.exe", 1),
"ExecutableArguments": OSError(2, "The system cannot find the file specified"),
},
78700704: {"SysVersion": ("3.6", 1), "SysArchitecture": ("magic", 1)},
78701824: {
"ExecutablePath": ("C:\\Users\\traveler\\AppData\\Local\\Programs\\Python\\Python36\\python.exe", 1),
"ExecutableArguments": OSError(2, "The system cannot find the file specified"),
},
78704368: {"SysVersion": ("3.6", 1), "SysArchitecture": (100, 4)},
78704048: {
"ExecutablePath": ("C:\\Users\\traveler\\AppData\\Local\\Programs\\Python\\Python36\\python.exe", 1),
"ExecutableArguments": OSError(2, "The system cannot find the file specified"),
},
78703024: {"SysVersion": ("3.6", 1), "SysArchitecture": ("64bit", 1)},
78701936: {
"ExecutablePath": OSError(2, "The system cannot find the file specified"),
None: OSError(2, "The system cannot find the file specified"),
},
78701792: {
"SysVersion": OSError(2, "The system cannot find the file specified"),
"SysArchitecture": OSError(2, "The system cannot find the file specified"),
},
78703792: {
"ExecutablePath": ("C:\\Users\\traveler\\AppData\\Local\\Programs\\Python\\Python35\\python.exe", 1),
"ExecutableArguments": OSError(2, "The system cannot find the file specified"),
},
78703424: {"SysVersion": ("3.6", 1), "SysArchitecture": ("64bit", 1)},
78701888: {
"ExecutablePath": ("C:\\Users\\traveler\\AppData\\Local\\Programs\\Python\\Python36\\python.exe", 1),
"ExecutableArguments": OSError(2, "The system cannot find the file specified"),
},
78704512: {"SysVersion": ("3.7", 1), "SysArchitecture": ("32bit", 1)},
78703600: {
"ExecutablePath": ("C:\\Users\\traveler\\AppData\\Local\\Programs\\Python\\Python37-32\\python.exe", 1),
"ExecutableArguments": OSError(2, "The system cannot find the file specified"),
},
78700656: {
"SysVersion": OSError(2, "The system cannot find the file specified"),
"SysArchitecture": OSError(2, "The system cannot find the file specified"),
},
78702608: {"SysVersion": ("magic", 1), "SysArchitecture": ("64bit", 1)},
78703632: {
"ExecutablePath": ("C:\\Users\\traveler\\AppData\\Local\\Programs\\Python\\Python36\\python.exe", 1),
"ExecutableArguments": OSError(2, "The system cannot find the file specified"),
},
78703088: {"SysVersion": (2778, 11)},
78703136: {
"SysVersion": OSError(2, "The system cannot find the file specified"),
"SysArchitecture": OSError(2, "The system cannot find the file specified"),
},
78700912: {
"ExecutablePath": OSError(2, "The system cannot find the file specified"),
None: ("C:\\Python27\\", 1),
"ExecutableArguments": OSError(2, "The system cannot find the file specified"),
},
78704032: {
"SysVersion": OSError(2, "The system cannot find the file specified"),
"SysArchitecture": OSError(2, "The system cannot find the file specified"),
},
78703648: {
"ExecutablePath": OSError(2, "The system cannot find the file specified"),
None: ("C:\\Python34\\", 1),
"ExecutableArguments": OSError(2, "The system cannot find the file specified"),
},
}
enum_collect = {
78701856: ["ContinuumAnalytics", "PythonCore", OSError(22, "No more data is available", None, 259, None)],
78701152: ["Anaconda37-32", "Anaconda37-64", OSError(22, "No more data is available", None, 259, None)],
78702656: [
"3.1",
"3.2",
"3.3",
"3.5",
"3.6",
"3.7-32",
"3.8",
"3.9",
"3.X",
OSError(22, "No more data is available", None, 259, None),
],
78701840: ["PyLauncher", "PythonCore", OSError(22, "No more data is available", None, 259, None)],
78702960: ["2.7", "3.4", OSError(22, "No more data is available", None, 259, None)],
}

25
tests/unit/interpreters/test_interpreters.py Normal file
View file

@ -0,0 +1,25 @@
from __future__ import absolute_import, unicode_literals
import sys
from uuid import uuid4
import pytest
from virtualenv.interpreters.discovery.py_info import CURRENT
from virtualenv.run import run_via_cli
def test_failed_to_find_bad_spec():
of_id = uuid4().hex
with pytest.raises(RuntimeError) as context:
run_via_cli(["-p", of_id])
msg = repr(RuntimeError("failed to find interpreter for Builtin discover of python_spec={!r}".format(of_id)))
assert repr(context.value) == msg
@pytest.mark.parametrize("of_id", [sys.executable, CURRENT.implementation])
def test_failed_to_find_implementation(of_id, mocker):
mocker.patch("virtualenv.run._collect_creators", return_value={})
with pytest.raises(RuntimeError) as context:
run_via_cli(["-p", of_id])
assert repr(context.value) == repr(RuntimeError("No virtualenv implementation for {}".format(CURRENT)))

13
tests/unit/test_run.py Normal file
View file

@ -0,0 +1,13 @@
import pytest
from virtualenv.run import run_via_cli
def test_help(capsys):
with pytest.raises(SystemExit) as context:
run_via_cli(args=["-h"])
assert context.value.code == 0
out, err = capsys.readouterr()
assert not err
assert out

47
tests/unit/test_util.py Normal file
View file

@ -0,0 +1,47 @@
from __future__ import absolute_import, unicode_literals
import logging
import os
import pytest
from virtualenv.util import run_cmd, symlink_or_copy
@pytest.mark.skipif(not hasattr(os, "symlink"), reason="requires symlink support")
def test_fallback_to_copy_if_symlink_fails(caplog, capsys, tmp_path, mocker):
caplog.set_level(logging.DEBUG)
mocker.patch("os.symlink", side_effect=OSError())
dst, src = _try_symlink(caplog, tmp_path, level=logging.WARNING)
msg = "symlink failed {!r}, for {} to {}, will try copy".format(OSError(), src, dst)
assert len(caplog.messages) == 1, caplog.text
message = caplog.messages[0]
assert msg == message
out, err = capsys.readouterr()
assert not out
assert err
def _try_symlink(caplog, tmp_path, level):
caplog.set_level(level)
src = tmp_path / "src"
src.write_text("a")
dst = tmp_path / "dst"
symlink_or_copy(do_copy=False, src=src, dst=dst)
assert dst.exists()
assert not dst.is_symlink()
assert dst.read_text() == "a"
return dst, src
@pytest.mark.skipif(hasattr(os, "symlink"), reason="requires no symlink")
def test_os_no_symlink_use_copy(caplog, tmp_path):
dst, src = _try_symlink(caplog, tmp_path, level=logging.DEBUG)
assert caplog.messages == ["copy {} to {}".format(src, dst)]
def test_run_fail(tmp_path):
code, out, err = run_cmd([str(tmp_path)])
assert err
assert not out
assert code

142
tox.ini Normal file
View file

@ -0,0 +1,142 @@
[tox]
minversion = 3.14.0
envlist =
fix_lint,
py38,
py37,
py36,
py35,
py34,
py27,
coverage
isolated_build = true
skip_missing_interpreters = true
[testenv]
description = run tests with {basepython}
deps =
pip >= 19.1.1
setenv =
COVERAGE_FILE = {toxworkdir}/.coverage.{envname}
COVERAGE_PROCESS_START = {toxinidir}/.coveragerc
_COVERAGE_SRC = {envsitepackagesdir}/virtualenv
passenv = https_proxy http_proxy no_proxy HOME PYTEST_* PIP_* CI_RUN TERM
extras = testing
install_command = python -m pip install {opts} {packages} --disable-pip-version-check
commands =
python -c 'from os.path import sep; file = open(r"{envsitepackagesdir}\{\}coverage-virtualenv.pth".format(sep), "w"); file.write("import coverage; coverage.process_startup()")'
coverage erase
coverage run\
-m pytest \
--junitxml {toxworkdir}/junit.{envname}.xml \
tests {posargs}
coverage combine
coverage report
[testenv:coverage]
description = [run locally after tests]: combine coverage data and create report;
generates a diff coverage against origin/master (can be changed by setting DIFF_AGAINST env var)
deps =
{[testenv]deps}
coverage >= 4.4.1, < 5
diff_cover
extras =
skip_install = True
passenv = DIFF_AGAINST
setenv =
COVERAGE_FILE={toxworkdir}/.coverage
commands =
coverage combine
coverage report --show-missing
coverage xml -o {toxworkdir}/coverage.xml
coverage html -d {toxworkdir}/htmlcov
diff-cover --compare-branch {env:DIFF_AGAINST:origin/rewrite} {toxworkdir}/coverage.xml
depends =
py38,
py37,
py36,
py35,
py34,
py27,
parallel_show_output = True
[testenv:docs]
basepython = python3.8
description = build documentation
extras = docs
commands =
sphinx-build -d "{envtmpdir}/doctree" -W docs "{toxworkdir}/docs_out" --color -bhtml {posargs}
python -c 'import pathlib; print("documentation available under file://\{0\}".format(pathlib.Path(r"{toxworkdir}") / "docs_out" / "index.html"))'
[testenv:package_readme]
description = check that the long description is valid (need for PyPi)
deps =
{[testenv]deps}
twine >= 1.12.1
skip_install = true
extras =
commands =
pip wheel -w {envtmpdir}/build --no-deps .
twine check {envtmpdir}/build/*
[testenv:upgrade]
description = upgrade pip/wheels/setuptools to latest
skip_install = true
deps =
pathlib2
black
passenv = UPGRADE_ADVISORY
changedir = {toxinidir}/tasks
commands = python upgrade_wheels.py
[testenv:fix_lint]
description = format the code base to adhere to our styles, and complain about what we cannot do automatically
basepython = python3.8
passenv = *
deps = {[testenv]deps}
pre-commit >= 1.17.0, <2
skip_install = True
commands =
pre-commit run --all-files --show-diff-on-failure
python -c 'import pathlib; print("hint: run \{\} install to add checks as pre-commit hook".format(pathlib.Path(r"{envdir}") / "bin" / "pre-commit"))'
[isort]
multi_line_output = 3
include_trailing_comma = True
force_grid_wrap = 0
line_length = 120
known_standard_library = ConfigParser
known_first_party = virtualenv
known_third_party = appdirs,coverage,entrypoints,git,packaging,pathlib2,pytest,setuptools,six
[flake8]
max-complexity = 22
max-line-length = 120
ignore = E203, W503, C901, E402
[pep8]
max-line-length = 120
[testenv:dev]
description = generate a DEV environment
extras = testing, docs
usedevelop = True
commands =
python -m pip list --format=columns
python -c 'import sys; print(sys.executable)'
[testenv:release]
description = do a release, required posarg of the version number
basepython = python3.8
skip_install = true
passenv = *
deps =
{[testenv]deps}
gitpython >= 2.1.10, < 3
towncrier >= 18.5.0
packaging >= 17.1
changedir = {toxinidir}/tasks
commands =
python release.py --version {posargs}