python-pytokens
Port variant v12
Summary Spec compliant tokenizer (3.12)
Package version 0.1.10
Homepage https://github.com/tusharsadhwani/pytokens
Keywords python
Maintainer Python Automaton
License Not yet specified
Other variants v13
Ravenports Buildsheet | History
Ravensource Port Directory | History
Last modified 07 OCT 2025, 02:22:54 UTC
Port created 07 OCT 2025, 02:22:54 UTC
Subpackage Descriptions
single # pytokens A Fast, spec compliant Python 3.12+ tokenizer that runs on older Pythons. ## Installation `bash pip install pytokens ` ## Usage ```bash pytokens path/to/file.py ``` ## Local Development / Testing - Create and activate a virtual environment - Run `pip install -r requirements-dev.txt` to do an editable install - Run `pytest` to run tests ## Type Checking Run `mypy .` ## Create and upload a package to PyPI Make sure to bump the version in `setup.cfg`. Then run the following commands: ```bash rm -rf dist python -m build ``` Then upload it to PyPI using [twine]: ```bash twine upload dist/* ```
Configuration Switches (platform-specific settings discarded)
PY312 ON Build using Python 3.12 PY313 OFF Build using Python 3.13
Package Dependencies by Type
Build (only) python312:dev:std
python-pip:single:v12
autoselect-python:single:std
Build and Runtime python312:primary:std
Download groups
main mirror://PYPIWHL/60/e5/63bed382f6a7a5ba70e7e132b8b7b8abbcf4888ffa6be4877698dcfbed7d
Distribution File Information
db7b72284e480e69fb085d9f251f66b3d2df8b7166059261258ff35f50fb711b 12046 python-src/pytokens-0.1.10-py3-none-any.whl
Ports that require python-pytokens:v12
python-black:v12 Uncompromising code formatter (3.12)