Comment by CurriedHautious
3 years ago
What is the compatibility of PyPy with a typical web server deployment? I am currently looking at testing compatibility with Tornado -> SQL Alchemy -> psycopg2. It seems like the C-extensions are a common tripping point. I see the recommendation to use psycopg2cffi, but it seems that package's last release was 2019 :(
SQL Alchemy actually points to PyPy in its recommendations of things to try in ORM performance. https://docs.sqlalchemy.org/en/20/faq/performance.html#resul...
The only compatibility issue for web development I've run into is database drivers.
For PostgreSQL, psycopg2 is not supported. psycopg2cffi is largely unmaintained, and the 2.9.0 version in PyPI lacks some newer features of psycopg2: the `psycopg2.sql` module and empty result sets raise a RuntimeError in Python 3.7+. The latest commit in on Github does have these changes [1]. Psycopg 3 [2] and pg8000 [3] (as user tlocke mentioned elsewhere) are viable alternates provided you aren't stuck with older versions of PostgreSQL. I have to continue use psycopg2cffi until I can upgrade an old PostgreSQL 9.4 database.
For Microsoft SQL Server, pymssql does not support PyPy [4]. It's under new maintainership so it might gain support in the future. pypyodbc hasn't had any activity since 2022, and no new PyPI release since 2021 [5]. The datatypes returned can differ between libodbc1 versions. On Ubuntu 18.04 in particular: empty string columns are returned as a single space, integer columns are returned as a Decimal. Also, if you encounter a mysterious HY010 error ("Function sequence error"), you may need to upgrade libodbc1 to v2.3.7+ from v2.3.4 using the Microsoft repos.
[1]: https://github.com/chtd/psycopg2cffi [2]: https://pypi.org/project/psycopg/ [3]: https://pypi.org/project/pg8000/ [4]: https://github.com/pymssql/pymssql/pull/517 [5]: https://pypi.org/project/pypyodbc/
Through some back & forth in conda and pip (fighting dependencies), I have been able to get PyPy 3.9 running with my ARM64 Debian system. So far I am seeing a performance decrease up to 2x. I have a series of REST API calls that encapsulate a single DB transaction - a mix of reads and writes. Most of it is leaning on SQL Alchemy, but we have been reaching for psycopg2 for some of the larger insert statements.
I was hoping to see some improvement in ORM performance (SQLAlchemy 1.3) - mainly in the bookkeeping side. Currently the app is about 60% Python app wait time and 40% DB wait time. We have a handful noisy areas which emit a lot of statements (Update 1 row at a time, 10000 times via ORM for example).
I also tried cProfiler to drill down, but as I've seen in Stack Overflow notes that profiler has a larger impact in PyPy over CPython.