Unable to include data files in python package with setuptools. (lemmy.world)

submitted 11 months ago* (last edited 11 months ago) by breadcrumb@lemmy.world to c/python@programming.dev

3 comments fedilink hide all child comments

I want to build a python package using setuptools. The folder structure of the project is the following (some non-essential parts have been deleted):

energy-monitor
├── config
│ ├── config.yml
│ └── secrets.yml
├── data
│ └── cpu_tdp.json
├── energy_monitor
│ ├── core
│ │ ├── gui.py
│ │ ├── __init__.py
│ ├── data
│ │ └── tableExport.json
│ ├── __init__.py
│ ├── main.py
│ └── utils
│     ├── api_calls.py
│     └── __init__.py
├── energy-monitor.py
├── LICENSE
├── MANIFEST.in
├── pyproject.toml
├── README.md
└── requirements.txt

The content of the pyproject.toml file is the following (some non-essential parts have been deleted):

[build-system]
requires = ["setuptools>=68.0"]
build-backend = "setuptools.build_meta"

[project]
name = "energy_monitor"
version = "0.0.1"
description = "Energy monitor"
readme = "README.md"
requires-python = ">=3.11"
license = {text = "GPLv3"}
classifiers = [
  "Programming Language :: Python :: 3",
  "Operating System :: OS Independent",
]
dynamic = ["dependencies"]

[tool.setuptools.dynamic]
dependencies = {file = ["requirements.txt"]}

[tool.setuptools]
packages = [
  "energy_monitor", 
  "energy_monitor.core", 
  "energy_monitor.utils"
]
include-package-data = true

[project.scripts]
energy-monitor = "energy_monitor.main:main"

Finally, the content of the MANIFEST.in file is the following:

include README.md
include LICENSE
graft config

I generate the package with python -m build and install the .tar.gz archive with pipx. According to setuptools documentation, I expect to find my config folder, together with README and LICENSE in the interpreter directory (site-packages) after the installation. However, this doesn't happen and I cannot run the app becase it complains that it doesn't find the config. What am I missing?

top 3 comments

sorted by: hot top controversial new old

[-] Andy@programming.dev 2 points 11 months ago

I don't have a proper answer, but if you are up to try ditching the setuptools backend, flit has great docs and I've never had a problem with it.

[-] NostraDavid@programming.dev 2 points 10 months ago* (last edited 10 months ago)

Doesn't MANIFEST.in specify which other-than-the-known-package files and folders to include?

I bet you need to add them to the MANIFEST.in.

I use poetry, so I don't need the MANIFEST.in, but I used it use a setup.py a few years ago, which is why I somewhat know of it.

https://packaging.python.org/en/latest/guides/using-manifest-in/

[-] UlrikHD@programming.dev 1 points 11 months ago* (last edited 11 months ago)

I don't have any experience with pipx and personally prefer to just skip the .toml and place the whole pyprojectsetup in setup.py.

With that method, I would write inside setup()

packages=find_packages()  # Include every python packages
package_data={  # Specify additional data files
    'yourpackagename': [
        'config/*'
        etc...
    ]
}

This would however require you to have a package folder which all your package files/folders are inside, meaning the top level repo folder should not have any files or other folders that you want to distribute. Your MANIFEST.in looks fine.