06-16, 15:45–16:25 (Europe/London), Warwick
Join us on a journey through the evolution of Python packaging, where we'll untangle the web of tools and formats that have shaped Python development. From the humble beginnings of Eggs to the sophisticated elegance of Poetry, this talk is a tribute to the ingenuity of the Python community. Along the way, we'll nod to Conda's cameo, recognising its unique contribution to making scientific packages more accessible. This story is for anyone who's ever wondered about the magic behind pip install and why packaging is so hard to get right in the Python world.
Outline:
- Introduction
Setting the stage by discussing the initial challenges of Python package management, a little bit about myself and how I embarked on this journey.
- Eggs and the Search for Structure
In the early 90s, Python developers, like their counterparts in other languages, grappled with the nascent chaos of software distribution. Languages like Perl had begun to consolidate around CPAN (Comprehensive Perl Archive Network), a centralized repository that significantly simplified package distribution and installation. This period marks Python's initial foray into packaging solutions, culminating in the creation of Python Eggs.
- Building Better: Setup.py, Setuptools, and the Introduction of Wheels
The evolution from simple scripts (setup.py) to the Setuptools project mirrored the broader software industry's move towards more robust build systems. As Java and .NET ecosystems developed intricate build and package management tools like Maven and NuGet, Python's introduction of Setuptools and later, Wheels, marked significant advancements in package installation and distribution efficiency. This chapter delves into the motivations behind these developments and their implications for Python packaging.
- Streamlining Development: pip and Virtualenv
The advent of pip as Python's package installer heralded a new era of convenience, akin to Ruby's RubyGems and Node.js's npm. Coupled with virtualenv, pip provided a seamless environment management and package installation experience, addressing the growing complexity of Python projects and their dependency trees. This segment explores how pip and virtualenv collectively elevated Python's development workflow.
- A Practical Note on Conda
Acknowledging Conda's role in the Python packaging story, especially for managing packages and environments that include non-Python dependencies. This brief segment will contextualize Conda's utility in scientific computing, emphasizing its complementary role rather than as a central focus.
Advancing the Ecosystem: pyenv, Pipenv, and pyproject.toml
- Discussing the evolution towards more sophisticated dependency management and project configuration, including the roles of pyenv, Pipenv, and pyproject.toml.
This part will focus on how these tools address specific challenges in scientific project management.
The Current Landscape: Poetry, PDM, and Their Place in Scientific Computing
Bringing the narrative to the present with an overview of Poetry and PDM. This section will evaluate these tools from a scientific perspective, considering their efficiency, ease of use, and how they fit into reproducible research workflows.
No previous knowledge expected
Nafiul Islam is a Software Engineer, Speaker and Author. With more than a decade of development experience, Nafiul loves talking about developer experience and how to make it better. Nafiul currently works at Sonar as the Developer Advocate for Python. Previously, he worked at JetBrains and Microsoft.