Commit 67f1323d authored by Suzanne Selhorn's avatar Suzanne Selhorn Committed by Craig Norris

Docs: Edited PyPI topic

Related to: https://gitlab.com/gitlab-org/gitlab/-/issues/247950
parent db2c7db2
...@@ -184,7 +184,7 @@ to authenticate with the API: ...@@ -184,7 +184,7 @@ to authenticate with the API:
- [Maven Repository](../user/packages/maven_repository/index.md#authenticate-with-a-ci-job-token-in-maven) - [Maven Repository](../user/packages/maven_repository/index.md#authenticate-with-a-ci-job-token-in-maven)
- [NPM Repository](../user/packages/npm_registry/index.md#authenticate-with-a-ci-job-token) - [NPM Repository](../user/packages/npm_registry/index.md#authenticate-with-a-ci-job-token)
- [Nuget Repository](../user/packages/nuget_repository/index.md) - [Nuget Repository](../user/packages/nuget_repository/index.md)
- [PyPI Repository](../user/packages/pypi_repository/index.md#publish-a-pypi-package-by-using-cicd) - [PyPI Repository](../user/packages/pypi_repository/index.md#authenticate-with-a-ci-job-token)
- [Generic packages](../user/packages/generic_packages/index.md#publish-a-generic-package-by-using-cicd) - [Generic packages](../user/packages/generic_packages/index.md#publish-a-generic-package-by-using-cicd)
- [Get job artifacts](job_artifacts.md#get-job-artifacts) - [Get job artifacts](job_artifacts.md#get-job-artifacts)
- [Pipeline triggers](pipeline_triggers.md) (using the `token=` parameter) - [Pipeline triggers](pipeline_triggers.md) (using the `token=` parameter)
......
...@@ -38,7 +38,6 @@ Learn more about using CI/CD to build: ...@@ -38,7 +38,6 @@ Learn more about using CI/CD to build:
- [Composer packages](../composer_repository/index.md#publish-a-composer-package-by-using-cicd) - [Composer packages](../composer_repository/index.md#publish-a-composer-package-by-using-cicd)
- [NuGet packages](../nuget_repository/index.md#publish-a-nuget-package-by-using-cicd) - [NuGet packages](../nuget_repository/index.md#publish-a-nuget-package-by-using-cicd)
- [Conan packages](../conan_repository/index.md#publish-a-conan-package-by-using-cicd) - [Conan packages](../conan_repository/index.md#publish-a-conan-package-by-using-cicd)
- [PyPI packages](../pypi_repository/index.md#publish-a-pypi-package-by-using-cicd)
- [Generic packages](../generic_packages/index.md#publish-a-generic-package-by-using-cicd) - [Generic packages](../generic_packages/index.md#publish-a-generic-package-by-using-cicd)
If you use CI/CD to build a package, extended activity information is displayed If you use CI/CD to build a package, extended activity information is displayed
......
...@@ -12,67 +12,75 @@ info: To determine the technical writer assigned to the Stage/Group associated w ...@@ -12,67 +12,75 @@ info: To determine the technical writer assigned to the Stage/Group associated w
Publish PyPI packages in your project’s Package Registry. Then install the Publish PyPI packages in your project’s Package Registry. Then install the
packages whenever you need to use them as a dependency. packages whenever you need to use them as a dependency.
The GitLab PyPI Repository works with: The Package Registry works with:
- [pip](https://pypi.org/project/pip/) - [pip](https://pypi.org/project/pip/)
- [twine](https://pypi.org/project/twine/) - [twine](https://pypi.org/project/twine/)
## Build a PyPI package ## Build a PyPI package
This section covers creating a new example PyPI package to upload. This is a This section explains how to create a PyPI package.
quickstart to test out the **GitLab PyPI Registry**. If you already understand
how to build and publish your own packages, move on to the [next section](#authenticate-with-the-package-registry). If you already use PyPI and know how to build your own packages, go to the
[next section](#authenticate-with-the-package-registry).
### Install pip and twine ### Install pip and twine
You need a recent version of [pip](https://pypi.org/project/pip/) and [twine](https://pypi.org/project/twine/). Install a recent version of [pip](https://pypi.org/project/pip/) and
[twine](https://pypi.org/project/twine/).
### Create a project ### Create a project
Understanding how to create a full Python project is outside the scope of this Create a test project.
guide, but you can create a small package to test out the registry. Start by
creating a new directory called `MyPyPiPackage`:
```shell 1. Open your terminal.
mkdir MyPyPiPackage && cd MyPyPiPackage 1. Create a directory called `MyPyPiPackage`, and then go to that directory:
```
After creating this, create another directory inside: ```shell
mkdir MyPyPiPackage && cd MyPyPiPackage
```
```shell 1. Create another directory and go to it:
mkdir mypypipackage && cd mypypipackage
```
Create two new files inside this directory to set up the basic project: ```shell
mkdir mypypipackage && cd mypypipackage
```
```shell 1. Create the required files in this directory:
touch __init__.py
touch greet.py ```shell
``` touch __init__.py
touch greet.py
```
Inside `greet.py`, add the following code: 1. Open the `greet.py` file, and then add:
```python ```python
def SayHello(): def SayHello():
print("Hello from MyPyPiPackage") print("Hello from MyPyPiPackage")
return return
``` ```
Inside the `__init__.py` file, add the following: 1. Open the `__init__.py` file, and then add:
```python ```python
from .greet import SayHello from .greet import SayHello
``` ```
Now that the basics of our project is completed, we can test that the code runs. 1. To test the code, in your `MyPyPiPackage` directory, start the Python prompt.
Start the Python prompt inside your top `MyPyPiPackage` directory. Then run:
```python ```shell
>>> from mypypipackage import SayHello python
>>> SayHello() ```
```
1. Run this command:
You should see an output similar to the following: ```python
>>> from mypypipackage import SayHello
>>> SayHello()
```
A message indicates that the project was set up successfully:
```plaintext ```plaintext
Python 3.8.2 (v3.8.2:7b3ab5921f, Feb 24 2020, 17:52:18) Python 3.8.2 (v3.8.2:7b3ab5921f, Feb 24 2020, 17:52:18)
...@@ -83,32 +91,30 @@ Type "help", "copyright", "credits" or "license" for more information. ...@@ -83,32 +91,30 @@ Type "help", "copyright", "credits" or "license" for more information.
Hello from MyPyPiPackage Hello from MyPyPiPackage
``` ```
After we've verified that the sample project is working as previously described,
we can next work on creating a package.
### Create a package ### Create a package
Inside your `MyPyPiPackage` directory, we need to create a `setup.py` file. Run After you create a project, you can create a package.
the following:
```shell 1. In your terminal, go to the `MyPyPiPackage` directory.
touch setup.py 1. Create a `setup.py` file:
```
```shell
touch setup.py
```
This file contains all the information about our package. For more information This file contains all the information about the package. For more information
about this file, see [creating setup.py](https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py). about this file, see [creating setup.py](https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py).
Becaue GitLab identifies packages based on Because GitLab identifies packages based on
[Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names), [Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names),
ensure your package name meets these requirements. See the [installation section](#publish-a-pypi-package-by-using-cicd) ensure your package name meets these requirements. See the [installation section](#authenticate-with-a-ci-job-token)
for more details. for details.
For this guide, we don't need to extensively fill out this file. Add the 1. Open the `setup.py` file, and then add basic information:
following to your `setup.py`:
```python ```python
import setuptools import setuptools
setuptools.setup( setuptools.setup(
name="mypypipackage", name="mypypipackage",
version="0.0.1", version="0.0.1",
author="Example Author", author="Example Author",
...@@ -121,45 +127,45 @@ setuptools.setup( ...@@ -121,45 +127,45 @@ setuptools.setup(
"Operating System :: OS Independent", "Operating System :: OS Independent",
], ],
python_requires='>=3.6', python_requires='>=3.6',
) )
``` ```
Save the file, and then execute the setup: 1. Save the file.
1. Execute the setup:
```shell ```shell
python3 setup.py sdist bdist_wheel python3 setup.py sdist bdist_wheel
``` ```
If successful, you should be able to see the output in a newly created `dist` The output should be visible in a newly-created `dist` folder:
folder. Run:
```shell ```shell
ls dist ls dist
``` ```
And confirm your output matches the below: The output should appear similar to the following:
```plaintext ```plaintext
mypypipackage-0.0.1-py3-none-any.whl mypypipackage-0.0.1.tar.gz mypypipackage-0.0.1-py3-none-any.whl mypypipackage-0.0.1.tar.gz
``` ```
The package is now all set up and is ready to be uploaded to the The package is now ready to be published to the Package Registry.
_GitLab PyPI Package Registry_. Before we do so, we next need to set up
authentication.
## Authenticate with the Package Registry ## Authenticate with the Package Registry
### Authenticate with a personal access token Before you can publish to the Package Registry, you must authenticate.
To do this, you can use:
You need the following: - A [personal access token](../../../user/profile/personal_access_tokens.md)
with the scope set to `api`.
- A [deploy token](./../../project/deploy_tokens/index.md) with the scope set to
`read_package_registry`, `write_package_registry`, or both.
- A [CI job token](#authenticate-with-a-ci-job-token).
- A personal access token. You can generate a ### Authenticate with a personal access token
[personal access token](../../../user/profile/personal_access_tokens.md)
with the scope set to `api` for repository authentication.
- A suitable name for your source.
- Your project ID, which is found on the home page of your project.
Edit your `~/.pypirc` file and add the following: To authenticate with a personal access token, edit the `~/.pypirc` file and add:
```ini ```ini
[distutils] [distutils]
...@@ -167,22 +173,17 @@ index-servers = ...@@ -167,22 +173,17 @@ index-servers =
gitlab gitlab
[gitlab] [gitlab]
repository = https://gitlab.com/api/v4/projects/<project_id>/packages/pypi repository = https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi
username = __token__ username = __token__
password = <your personal access token> password = <your personal access token>
``` ```
### Authenticate with a deploy token - `username` must be `__token__` exactly.
- Your project ID is on your project's home page.
You need the following:
- A deploy token. You can generate a [deploy token](./../../project/deploy_tokens/index.md) ### Authenticate with a deploy token
with the `read_package_registry` or `write_package_registry` scopes for
repository authentication.
- A suitable name for your source.
- Your project ID, which is found on the home page of your project.
Edit your `~/.pypirc` file and add the following: To authenticate with a deploy token, edit your `~/.pypirc` file and add:
```ini ```ini
[distutils] [distutils]
...@@ -190,22 +191,57 @@ index-servers = ...@@ -190,22 +191,57 @@ index-servers =
gitlab gitlab
[gitlab] [gitlab]
repository = https://gitlab.com/api/v4/projects/<project_id>/packages/pypi repository = https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi
username = <deploy token username> username = <deploy token username>
password = <deploy token> password = <deploy token>
``` ```
Your project ID is on your project's home page.
### Authenticate with a CI job token
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202012) in GitLab 13.4.
To work with PyPI commands within [GitLab CI/CD](./../../../ci/README.md), you
can use `CI_JOB_TOKEN` instead of a personal access token or deploy token.
For example:
```yaml
image: python:latest
run:
script:
- pip install twine
- python setup.py sdist bdist_wheel
- TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url https://gitlab.example.com/api/v4/projects/${CI_PROJECT_ID}/packages/pypi dist/*
```
You can also use `CI_JOB_TOKEN` in a `~/.pypirc` file that you check in to
GitLab:
```ini
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.example.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/pypi
username = gitlab-ci-token
password = ${env.CI_JOB_TOKEN}
```
## Publish a PyPI package ## Publish a PyPI package
When publishing packages, note that: When publishing packages, note that:
- The maximum allowed size is 50 Megabytes. - The maximum allowed size is 50 MB.
- You can't upload the same version of a package multiple times. If you try, - You can't upload the same version of a package multiple times. If you try,
you'll receive the error `Validation failed: File name has already been taken`. you'll receive the error `Validation failed: File name has already been taken`.
### Ensure your version string is valid ### Ensure your version string is valid
If your version string (for example, `0.0.1`) is invalid, it will be rejected. If your version string (for example, `0.0.1`) isn't valid, it will be rejected.
GitLab uses the following regex to validate the version string. GitLab uses the following regex to validate the version string.
```ruby ```ruby
...@@ -220,120 +256,86 @@ GitLab uses the following regex to validate the version string. ...@@ -220,120 +256,86 @@ GitLab uses the following regex to validate the version string.
)\z}xi )\z}xi
``` ```
You can experiment with the regex and try your version strings using this You can experiment with the regex and try your version strings by using this
[regular expression editor](https://rubular.com/r/FKM6d07ouoDaFV). [regular expression editor](https://rubular.com/r/FKM6d07ouoDaFV).
For more details about the regex used, review this [documentation](https://www.python.org/dev/peps/pep-0440/#appendix-b-parsing-version-strings-with-regular-expressions). For more details about the regex, review this [documentation](https://www.python.org/dev/peps/pep-0440/#appendix-b-parsing-version-strings-with-regular-expressions).
### Publish a PyPI package by using twine ### Publish a PyPI package by using twine
If you were following the steps on this page, the `MyPyPiPackage` package should To publish a PyPI package, run a command like:
be ready to be uploaded. Run the following command:
```shell ```shell
python3 -m twine upload --repository gitlab dist/* python3 -m twine upload --repository gitlab dist/*
``` ```
If successful, you should see the following: This message indicates that the package was published successfully:
```plaintext ```plaintext
Uploading distributions to https://gitlab.com/api/v4/projects/<your_project_id>/packages/pypi Uploading distributions to https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi
Uploading mypypipackage-0.0.1-py3-none-any.whl Uploading mypypipackage-0.0.1-py3-none-any.whl
100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.58k/4.58k [00:00<00:00, 10.9kB/s] 100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.58k/4.58k [00:00<00:00, 10.9kB/s]
Uploading mypypipackage-0.0.1.tar.gz Uploading mypypipackage-0.0.1.tar.gz
100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.24k/4.24k [00:00<00:00, 11.0kB/s] 100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.24k/4.24k [00:00<00:00, 11.0kB/s]
``` ```
This indicates that the package was uploaded successfully. You can then navigate To view the published package, go to your project's **Packages & Registries**
to your project's **Packages & Registries** page and see the uploaded packages. page.
If you would rather not use a `.pypirc` file to define your repository source, If you didn't use a `.pypirc` file to define your repository source, you can
you can upload to the repository with the authentication inline: publish to the repository with the authentication inline:
```shell ```shell
TWINE_PASSWORD=<personal_access_token or deploy_token> TWINE_USERNAME=<username or deploy_token_username> python3 -m twine upload --repository-url https://gitlab.com/api/v4/projects/<project_id>/packages/pypi dist/* TWINE_PASSWORD=<personal_access_token or deploy_token> TWINE_USERNAME=<username or deploy_token_username> python3 -m twine upload --repository-url https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi dist/*
``` ```
If you didn't use the steps on this page, you need to ensure your package has If you didn't follow the steps on this page, ensure your package was properly
been properly built, and that you [created a PyPI package with `setuptools`](https://packaging.python.org/tutorials/packaging-projects/). built, and that you [created a PyPI package with `setuptools`](https://packaging.python.org/tutorials/packaging-projects/).
You can then upload your package using the following command: You can then upload your package by using the following command:
```shell ```shell
python -m twine upload --repository <source_name> dist/<package_file> python -m twine upload --repository <source_name> dist/<package_file>
``` ```
Where:
- `<package_file>` is your package filename, ending in `.tar.gz` or `.whl`. - `<package_file>` is your package filename, ending in `.tar.gz` or `.whl`.
- `<source_name>` is the [source name used during setup](#authenticate-with-the-package-registry). - `<source_name>` is the [source name used during setup](#authenticate-with-the-package-registry).
### Publish a PyPI package by using CI/CD
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202012) in GitLab 13.4.
To work with PyPI commands within [GitLab CI/CD](./../../../ci/README.md), you
can use `CI_JOB_TOKEN` in place of the personal access token or deploy a token
in your commands.
For example:
```yaml
image: python:latest
run:
script:
- pip install twine
- python setup.py sdist bdist_wheel
- TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/pypi dist/*
```
You can also use `CI_JOB_TOKEN` in a `~/.pypirc` file that you check into GitLab:
```ini
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/pypi
username = gitlab-ci-token
password = ${env.CI_JOB_TOKEN}
```
## Install a PyPI package ## Install a PyPI package
Install the latest version of a package using the following command: To install the latest version of a package, use the following command:
```shell ```shell
pip install --extra-index-url https://__token__:<personal_access_token>@gitlab.com/api/v4/projects/<project_id>/packages/pypi/simple --no-deps <package_name> pip install --extra-index-url https://__token__:<personal_access_token>@gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple --no-deps <package_name>
``` ```
Where:
- `<package_name>` is the package name. - `<package_name>` is the package name.
- `<personal_access_token>` is a personal access token with the `read_api` scope. - `<personal_access_token>` is a personal access token with the `read_api` scope.
- `<project_id>` is the project ID. - `<project_id>` is the project ID.
If you were following the guide above and want to test installing the If you were following the guide and want to install the
`MyPyPiPackage` package, you can run the following: `MyPyPiPackage` package, you can run:
```shell ```shell
pip install mypypipackage --no-deps --extra-index-url https://__token__:<personal_access_token>@gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/simple pip install mypypipackage --no-deps --extra-index-url https://__token__:<personal_access_token>@gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/simple
``` ```
This should result in the following: This message indicates that the package was installed successfully:
```plaintext ```plaintext
Looking in indexes: https://__token__:****@gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/simple Looking in indexes: https://__token__:****@gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/simple
Collecting mypypipackage Collecting mypypipackage
Downloading https://gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/files/d53334205552a355fee8ca35a164512ef7334f33d309e60240d57073ee4386e6/mypypipackage-0.0.1-py3-none-any.whl (1.6 kB) Downloading https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/files/d53334205552a355fee8ca35a164512ef7334f33d309e60240d57073ee4386e6/mypypipackage-0.0.1-py3-none-any.whl (1.6 kB)
Installing collected packages: mypypipackage Installing collected packages: mypypipackage
Successfully installed mypypipackage-0.0.1 Successfully installed mypypipackage-0.0.1
``` ```
GitLab looks for packages using ### Package names
[Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names),
so the characters `-`, `_`, and `.` are all treated the same and repeated characters are removed. GitLab looks for packages that use
[Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names).
The characters `-`, `_`, and `.` are all treated the same, and repeated
characters are removed.
A `pip install` request for `my.package` looks for packages that match any of A `pip install` request for `my.package` looks for packages that match any of
the three characters, such as `my-package`, `my_package`, and `my....package`. the three characters, such as `my-package`, `my_package`, and `my....package`.
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment