python-pytokens
Port variant v12
Summary Spec compliant tokenizer (3.12)
Package version 0.3.0
Homepage https://github.com/tusharsadhwani/pytokens
Keywords python
Maintainer Python Automaton
License Not yet specified
Other variants v13
Ravenports Buildsheet | History
Ravensource Port Directory | History
Last modified 05 NOV 2025, 21:49:58 UTC
Port created 07 OCT 2025, 02:22:54 UTC
Subpackage Descriptions
single # pytokens A Fast, spec compliant Python 3.14+ tokenizer that runs on older Pythons. ## Installation `bash pip install pytokens ` ## Usage ```bash python -m pytokens path/to/file.py ``` ## Local Development / Testing - Create and activate a virtual environment - Run `pip install -r requirements-dev.txt` to do an editable install - Run `pytest` to run tests ## Type Checking Run `mypy .` ## Create and upload a package to PyPI Make sure to bump the version in `setup.cfg`. Then run the following commands: ```bash rm -rf dist python -m build ``` Then upload it to PyPI using [twine]: ```bash twine upload dist/* ```
Configuration Switches (platform-specific settings discarded)
PY312 ON Build using Python 3.12 PY313 OFF Build using Python 3.13
Package Dependencies by Type
Build (only) python312:dev:std
python-pip:single:v12
autoselect-python:single:std
Build and Runtime python312:primary:std
Download groups
main mirror://PYPIWHL/84/25/d9db8be44e205a124f6c98bc0324b2bb149b7431c53877fc6d1038dddaf5
Distribution File Information
95b2b5eaf832e469d141a378872480ede3f251a5a5041b8ec6e581d3ac71bbf3 12195 python-src/pytokens-0.3.0-py3-none-any.whl
Ports that require python-pytokens:v12
python-black:v12 Uncompromising code formatter (3.12)