Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
154 changes: 154 additions & 0 deletions docs/wiki-guide/GitHub-PyPI-Zenodo-Integration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
# PyPI & Zenodo Release Automation

This page is a quick reference for setting up a repository to automatically release to the [Python Package Index (PyPI)](https://pypi.org/) and [Zenodo](https://zenodo.org/) on a GitHub release. At the bottom of the page are [checklists](#release-checklist) for first-time releases, new releases, and post-release steps. These are set up to be copied into a relevant issue on your GitHub repository for easier tracking.

!!! warning
Before setting a release, please review the [Digital Product Policy](About-Digital-Product-Policies.md) and check your repository against the [Code Repo Checklist](Code-Checklist.md).

!!! note
Steps marked with an `*` need only be completed once (the first time this is set up).

## Before Generating a Release

### Set version in... <!-- markdownlint-disable-line MD026 -->

- `__about__.py` file
- Version should be set *dynamically* in the `pyproject.toml` from this file.*
- [`CITATION.cff`](GitHub-Repo-Guide.md#citation-templates)
- [`.zenodo.json`](GitHub-Repo-Guide.md#zenodo-metadata)

### Add/Update Citation and Zenodo Metadata Files

Review or create both a [`CITATION.cff`](GitHub-Repo-Guide.md#citation) and [`.zenodo.json`](GitHub-Repo-Guide.md#zenodo-metadata) file following guide instructions. Be sure to include

!!! tip "Pro tip"
Add this [test to validate the Zenodo metatdata file](https://github.com/Imageomics/Collaborative-distributed-science-guide/blob/main/.github/workflows/validate-zenodo.yaml). It will run whenever the `.zenodo.json` or workflow file are edited, ensuring that your Zenodo integration will run smoothly.

Before release, ensure that...

- All authors are included and appropriately ordered.
- This list may or may not change, and may differ from an associated paper author list, as discussed in the [Digital Product Lifecycle](Digital-Product-Lifecycle.md#exploration-phase) and the [Imageomics Author Guide](https://docs.google.com/spreadsheets/d/1GwlCukfoQPL8JI2yyWRD3g4uiMTO3tlGNE_qeb_xBCs/edit?usp=sharing).
- All project keywords are listed, using quotes for multi-word tags.
- The release date in both files is correct.

!!! info
This should be added in a PR as the last commit before a release, but check that the date matches, since these PRs may not necessarily be merged on the same day they are created.

- Set [citation file identifiers](GitHub-Repo-Guide.md#__codelineno-1-8):
- Version tag link should match the updated version. The commit hash of the version will be filled in *after* release.
- DOI should be the *general software* DOI from Zenodo to avoid mis-matched version confusion (since it must be added ***after*** release).*

!!! note
If you have not generated a release yet, then leave the `doi` key commented out; it will be added **after** release.

### Automate publish to PyPI on release*

Complete all steps below to automatically publish releases to PyPI. These steps need only be completed once before the first release; only dependencies and other keys in the `pyproject.toml` may need updating over time.

- Set up a [`pyproject.toml`](https://packaging.python.org/en/latest/guides/writing-pyproject-toml/) with your project repo information and dependencies.
- Include your project keywords in `pyproject.toml` (no spaces, even within quotes).
- Version in `pyproject.toml` set *dynamically*.
- Add PyPI publication workflow*. See [pybioclip workflow](https://github.com/Imageomics/pybioclip/blob/main/.github/workflows/publish-to-pypi.yml) as an example.[^1]
- For this to work on upload, you must configure GitHub as a "pending" publisher on PyPI. See PyPI's ["pending" publisher docs](https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/) for instructions.

### Update `README` Installation Instructions*

Update the installation instructions in the `README` to `pip install <package-name>` instead of `pip install git+<GitHub URL>` right before the release and concurrent PyPI publication.

### Turn Repository Sync on in your Zenodo Account*

This must be set specifically for the target repo. See [Automatic DOI Generation with Zenodo](DOI-Generation.md#automatic-generation) for more information.

## After a Release is Generated

There are a few updates required after generating a release, though the last listed here is the only one that is required for each release.

- Add DOI badge to `README`*.
- Make sure to use the *version-agnostic* DOI, used to reference all releases. The badge display will then auto-update with each release to match the DOI of *that* release.
- See [Add a Zenodo DOI Badge](DOI-Generation.md#add-a-zenodo-doi-badge) for instructions.
- Add PyPI badges to `README`*.
- Add *version-agnostic* DOI to `CITATION.cff`*.
- Add release commit hash to `CITATION.cff` [identifiers](GitHub-Repo-Guide.md#__codelineno-1-8).

## Release Checklist

!!! tip "Pro tip"

Copy the markdown for the relevant checklist below into an issue on your GitHub [Repo](GitHub-Repo-Guide.md) or [Project](Guide-to-GitHub-Projects.md) so you can check the boxes as you complete each step to genearate a release for your GitHub repository.

=== "First Release"

```Markdown
### Set version in...
- [ ] `__about__.py` file (the only repeated PyPI update).
- Version should be set *dynamically* in the `pyproject.toml` from this file.*
- [ ] [`CITATION.cff`](https://imageomics.github.io/Imageomics-guide/wiki-guide/GitHub-Repo-Guide/#citation-templates).
- [ ] [`.zenodo.json`](https://imageomics.github.io/Imageomics-guide/wiki-guide/GitHub-Repo-Guide/#zenodo-metadata).

### Add/Update the [citation](https://imageomics.github.io/Imageomics-guide/wiki-guide/GitHub-Repo-Guide/#citation) and [Zenodo metadata](https://imageomics.github.io/Imageomics-guide/wiki-guide/GitHub-Repo-Guide/#zenodo-metadata) files.

Ensure that...
- [ ] All authors are included and appropriately ordered.
- This list may or may not change, and may differ from an associated paper author list, as discussed in the [Digital Product Lifecycle](https://imageomics.github.io/Imageomics-guide/wiki-guide/Digital-Product-Lifecycle/#exploration-phase) and the [Imageomics Author Guide](https://docs.google.com/spreadsheets/d/1GwlCukfoQPL8JI2yyWRD3g4uiMTO3tlGNE_qeb_xBCs/edit?usp=sharing).
- [ ] All project keywords are listed, using quotes for multi-word tags.
- [ ] The release date in both files is correct.
> [!IMPORTANT]
> This should be added in a PR as the last commit before a release, but check that the date matches, since these PRs may not necessarily be merged on the same day they are created.
- Set [citation file identifiers](https://imageomics.github.io/Imageomics-guide/wiki-guide/GitHub-Repo-Guide/#__codelineno-1-8):
- [ ] Version tag link should match the updated version. The commit hash of the version will be filled in *after* release.
> [!NOTE]
> You have not generated a release yet, so leave the `doi` key commented out; it will be added **after** release.
- [ ] Add [test to validate Zenodo file](https://github.com/Imageomics/Imageomics-guide/blob/main/.github/workflows/validate-zenodo.yaml).*

### Automate publish to PyPI on release:
- [ ] Set up a [`pyproject.toml`](https://packaging.python.org/en/latest/guides/writing-pyproject-toml/) with your project repo information and dependencies.
- [ ] Include your project keywords in `pyproject.toml` (no spaces, even within quotes).
- [ ] Version in `pyproject.toml` set *dynamically*.
- [ ] Add PyPI publication workflow*. See [pybioclip workflow](https://github.com/Imageomics/pybioclip/blob/main/.github/workflows/publish-to-pypi.yml) as an example.[^1]
- [ ] Configure GitHub as a "pending" publisher on PyPI. See PyPI's ["pending" publisher docs](https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/) for instructions.*

### Update installation instructions to reflect imminent release and PyPI publication:*
- [ ] `pip install <package-name>` instead of `pip install git+<GitHub URL>`.

### Zenodo Sync
- [ ] Turn on repository sync in your Zenodo account; this must be set specifically for the target repo. See [Automatic DOI Generation with Zenodo](https://imageomics.github.io/Imageomics-guide/wiki-guide/DOI-Generation/#automatic-generation) for more information.
```

=== "New Release"

```Markdown
### Set version in...
- [ ] `__about__.py` file (the only repeated PyPI update).
- Version should be set *dynamically* in the `pyproject.toml` from this file.*
- [ ] [`CITATION.cff`](https://imageomics.github.io/Imageomics-guide/wiki-guide/GitHub-Repo-Guide/#citation-templates).
- [ ] [`.zenodo.json`](https://imageomics.github.io/Imageomics-guide/wiki-guide/GitHub-Repo-Guide/#zenodo-metadata).

### Update the [citation](https://imageomics.github.io/Imageomics-guide/wiki-guide/GitHub-Repo-Guide/#citation) and [Zenodo metadata](https://imageomics.github.io/Imageomics-guide/wiki-guide/GitHub-Repo-Guide/#zenodo-metadata) files.

Ensure that...
- [ ] All authors are included and appropriately ordered.
- [ ] All project keywords are listed, using quotes for multi-word tags.
- [ ] The release date in both files is correct.

> [!IMPORTANT]
> This should be added in a PR as the last commit before a release, but check that the date matches, since these PRs sometimes are not merged on the day they are created.

- [ ] Set version tag in [citation file identifiers](https://imageomics.github.io/Imageomics-guide/wiki-guide/GitHub-Repo-Guide/#__codelineno-1-8) to match the updated version. The commit hash of the version will be filled in *after* release.
```

=== "After First Release"
```Markdown
- [ ] Add *version-agnostic* DOI badge to `README`*.
- See [Add a Zenodo DOI Badge](https://imageomics.github.io/Imageomics-guide/wiki-guide/DOI-Generation/#add-a-zenodo-doi-badge) for instructions.
- [ ] Add PyPI badges to `README`*.
- [ ] Add *version-agnostic* DOI to `CITATION.cff`*.
- [ ] Add release commit hash to `CITATION.cff` [identifiers](https://imageomics.github.io/Imageomics-guide/wiki-guide/GitHub-Repo-Guide/#__codelineno-1-8).
```

=== "After Release"

```Markdown
- [ ] Add release commit hash to `CITATION.cff` [identifiers](https://imageomics.github.io/Imageomics-guide/wiki-guide/GitHub-Repo-Guide/#__codelineno-1-8).
```

[^1]: Note that this follows the [PyPI Trusted Publishers docs](https://docs.pypi.org/trusted-publishers/using-a-publisher/) to remove the need for a token, though you must first configure GitHub as a ["pending" Trusted Publisher](https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/) for the project on PyPI.
5 changes: 5 additions & 0 deletions docs/wiki-guide/Handling-API-Keys.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ If you are using a web service with API keys, there are a few things to keep in
- Convenient to change if needed
- Unique for different environments

!!! danger "Protect your API Keys and Access Tokens!"
The most important lesson in key-management is to ***never make them public*** and ***always scope access***. If you think a key has been exposed, ***disable*** and ***regenerate*** it immediately. By keeping access keys and tokens scoped to a single task or repository with the least permissions necessary to complete the needed function, you further reduce the potential impact of a leaked key.

Both GitHub and Hugging Face recommend using *fine-grained* tokens, and provide guidance on creating and managing them within the platforms. Learn more about platform-specific access tokens on [GitHub](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) and [Hugging Face](https://huggingface.co/docs/hub/en/security-tokens). Keep reading below for more general API Key management guidance.

## Key Storage

Our recommended way of storing and using API is within `.env` (dotenv) files.
Expand Down
47 changes: 32 additions & 15 deletions docs/wiki-guide/The-GitHub-Pull-Request-Guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,36 +147,53 @@ Navigate to your repository name, click **Pull requests**

### 3.2 Incorporate Feedback Changes

After receiving feedback on your pull request, you can apply the changes in one of two ways: either by committing each change individually or by grouping several changes into a single commit. The method you choose depends on whether you prefer fine-grained control over the commit history or a more streamlined approach.
After receiving feedback on your pull request, you can apply the changes in one of two ways: either by [committing each change individually](#321-apply-a-change-in-its-own-commit) or by [grouping several changes into a single commit](#322-add-multiple-suggestions-to-a-batch-of-changes). The method you choose depends on whether you prefer fine-grained control over the commit history or a more streamlined approach, which also allows for grouping related changes for easier review.

#### 3.2.1 Apply a change in its own commit

If you agree with at suggested change, qpply it by creating a separate commit for it. This approach helps keep your commit history clear and each change traceable.
If you agree with a suggested change, apply it in the "Conversation" tab by selecting "Commit suggestion" to create a separate commit for it. This approach helps keep your commit history clear and each change traceable.

![Commit suggestions button](images/GH-PR-guide/369920474-be5503d3-6cc2-4313-b49d-069a5a806ac4.png){ loading=lazy, width="600" }
/// caption
///
<!-- Caption doesn't seem to have the fine-grained controls for smaller text spanning the width of the image, so bypassing HTML rule to allow for desired formatting. -->
<!-- markdownlint-disable MD033 -->
![Commit suggestions button highlighted with dropdown/pop-up box with the text prompt to add a commit message and the 'Commit suggestions' button at the bottom right](images/GH-PR-guide/369920474-be5503d3-6cc2-4313-b49d-069a5a806ac4.png){ loading=lazy, style="width:100%" }
<small>This button will bring up a "Commit suggestions" box in which you can update the commit message with something informative about *why* this change is being made. Please see the [GitHub Repo Guide section on commits](GitHub-Repo-Guide.md#commits) for more guidance on how to write a good commit message.</small>
<!-- markdownlint-enable MD033 -->

#### 3.2.2 Add multiple suggestions to a batch of changes

If you plan to include multiple changes in one commit, you can add suggestions to a batch. Once you've collected all the desired suggestions, click "Commit suggestions" to apply them in one go.
If you plan to include multiple changes in one commit, you can add suggestions to a batch. This is the preferred method when multiple suggestions are needed to implement the full change. First, navigate to the "Files changed" tab.

![Add suggested change to batch button](images/GH-PR-guide/369920952-1b4e0db0-3451-448b-822f-dd1b14679ec6.png){ loading=lazy, width="600" }
![Pull request header section with different view tabs, the 'Files changed' tab is selected with a box around it to highlight where it is](images/GH-PR-guide/Files-changed-tab.png){ loading=lazy, style="width:100%" }
/// caption
///

### 3.3 Add Commit Message
As you scroll through the updated files, comments and suggestions will be displayed at the line(s) to which they refer. Select the option to "Add suggestion to batch" for all suggestions you wish to include; use the "Remove from batch" option to remove those you do not want to include.

![Add suggested change to batch button](images/GH-PR-guide/Add-to-batch.png){ loading=lazy, style="width:45.5%" }
![remove suggested change from batch button](images/GH-PR-guide/Remove-from-batch.png){ loading=lazy, style="width:45%" }

Once you've collected all the desired suggestions, click "Commit suggestions" at the top of the tab, then select "Apply suggestions", to commit them all in one go. Observe that the number of selected suggestions appears in both these buttons (5 in this example). All the suggestions to commit can be scrolled through in this pop-up.

![Commit suggestions button is green on the right, with the words 'Commit suggestions 5' on it, a 'Discard batch' option is to its left, the comments button with the number 5 is all the way to the right inline with them](images/GH-PR-guide/Commit-suggestions.png){ loading=lazy, style="float:right;width:75%" }
![Apply suggestions pop-up window, the button is green and in the bottom right, with the words 'Apply suggestions 5' on it, suggestions are available to scroll through, and 'Discard batch' is bottom left button, there's also a 'Cancel' button to the left of the 'Apply suggestions' button](images/GH-PR-guide/Apply-suggestions.png){ loading=lazy, style="width:75%" }

#### 3.2.3 Add Commit Message

In the commit message field, enter a brief, descriptive message that clearly explains *the reasoning for the changes* made to the file(s). For instance, the above changes might best be served by the following commit message, since a function's behavior has been changed.
> Change expected output to match new function return

Please see the [GitHub Repo Guide section on commits](GitHub-Repo-Guide.md#commits) for more guidance on how to write a good commit message.

In the commit message field, enter a brief, descriptive message that clearly explains the changes made to the file(s).
#### 3.2.4 Commit changes

### 3.4 Click Commit changes
After entering your commit message, click the "Apply Suggestions" button to finalize the commit and thus save your modifications to the repository. This step ensures that your changes are recorded and can be reviewed or merged into the main codebase.

After entering your commit message, click the "Commit changes" button to finalize and save your modifications to the repository. This step ensures that your changes are recorded and can be reviewed or merged into the main codebase.
### 3.4 Re-requesting a Review

### 3.5 Re-requesting a Review
Once you have addressed all the requested changes, click the "re-request review" button to notify the PR reviewer(s). This action prompts them to evaluate your updated code and provide feedback or approval.

If you’ve addressed all the requested changes and your pull request requires further review, re-request a review by notifying the reviewers. This action prompts them to evaluate your updated code and provide feedback or approval.
![Re-request review is the hover text for the blue arrows forming a circle pointing at each other in the 'Reviewers' section of the side panel](images/GH-PR-guide/re-request-review.png){ loading=lazy, style="width:60%" }

### 3.6 Out-of-scope Suggestion
### 3.5 Out-of-scope Suggestion

If the suggested change falls outside the scope of your pull request, create a new issue to address the feedback separately. Issues can be created directly from a PR comment.
If the suggested change falls outside the scope of your pull request, create a new issue to address the feedback separately. Issues can be created directly from a PR comment by clicking on the three dots in the top right corner of the comment and selecting "Reference in new issue".
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 3 additions & 1 deletion mkdocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,9 @@ nav:
- "About Templates": wiki-guide/About-Templates.md
- "Dataset Card Template": wiki-guide/HF_DatasetCard_Template_mkdocs.md
- "Model Card Template": wiki-guide/HF_ModelCard_Template_mkdocs.md
- Command Line Cheat Sheet: wiki-guide/Command-Line-Cheat-Sheet.md
- Quick References:
- "Command Line Cheat Sheet": wiki-guide/Command-Line-Cheat-Sheet.md
- "PyPI & Zenodo Release Automation": wiki-guide/GitHub-PyPI-Zenodo-Integration.md
- Code of Conduct: CODE_OF_CONDUCT.md
- Digital Product Policy:
- "About Digital Product Policies": wiki-guide/About-Digital-Product-Policies.md
Expand Down
Loading