Delivering Open Source Expertise to the Enterprise With Our Quansight Labs Partnership

Picture of Quansight Staff

Quansight Staff

Image of a Quansight Labs Annual Report 2023 mockup.

In the recently released Quansight Labs Annual Report 2023, the Labs team did a deep dive into their progress including detailing their contributions to the open source ecosystem. Last year alone, they committed 38,000+ hours to 35+ open source projects, including NumPy, Pandas, SciPy, Matplotlib, scikit-learn, PyTorch, and JupyterLab. Their contributions also extended beyond core libraries, reaching into areas such as high-performance computing, visualization, Python packaging, and infrastructure tools.

Quansight Labs’ expertise, hard work, and dedication are a large piece of the puzzle for Quansight consulting and our unique ability to build best-of-class client-solutions on a foundation of strong open source projects.

Quansight, Quansight Labs & Quansight Initiate—What’s the Difference?

Three approaches to a common goal: A thriving open source ecosystem that helps organizations make better data-derived decisions.

Quansight’s mission is to create insights from quantitative data to enable better decisions on a foundation of sustainable open source. We have three “arms” that work toward this end with their own separate goals and leadership: Quansight Initiate, Quansight Labs, and Quansight (Consulting).

Quansight Initiate is a venture fund investing in data-heavy companies that use and contribute back to open source. It is investing out of its 2nd fund currently and plans to raise a larger fund in the near future. You can learn more at QI Ventures, or by checking out OpenTeams Incubator, which is a related organization we have retained to manage the fund. At least 15% of the proceeds from Quansight Initiate (to the general partner) will benefit our open source initiatives at Quansight and Quansight Labs. 

Quansight Labs is a break-even, open source R&D lab that focuses on creating a core team for the SciPy and PyData ecosystems. It was created in 2019 and the team consists of developers, community leads, designers, and writers. Labs is at the forefront of sustaining and advancing open source projects critical to the data science ecosystem and that symbiotic relationship between open source contributions and client solutions is a cornerstone of Quansight’s delivery approach. All of the engineers on the Labs team spend some time on consulting projects throughout the year to keep the real-world application of open source fresh in their minds. Additionally, they’re allotted time to work on the open source projects they maintain or are passionate about. This model has proven successful and ensures engineers are able to work on what they care about while gaining invaluable knowledge that our customers benefit from when they engage with us to build custom solutions.

Quansight Labs is a foundation on which Quansight (Consulting) supports our clients to build their own software and systems that help their particular business and sustain their own vision of the future. Quansight can help anyone using the SciPy and PyData ecosystem to solve a problem, but we have done a lot of work helping companies build best-of-breed data-, ML-, and AI- engineering and operation solutions. Our customers benefit significantly from our experienced and educated team of architects, senior developers, and domain experts. Our open source experience and tight connection with Labs also makes us the perfect partner to help build our client’s open source strategy or systems.

Some of the open source projects we maintain and contribute to.

From the Report: Quansight Labs Project Highlights 2023

The mission of Quansight Labs is to sustain and grow community-driven open source projects and ecosystems, focusing on the core of the PyData stack and tools and digital infrastructure for data science, scientific computing, Machine Learning (ML), and Artificial Intelligence (AI). Millions of people worldwide use open source software and infrastructure every day.

To see the complete list of project highlights, read the Quansight Labs Annual Report 2023. At a high level here’s some of what the Labs team accomplished last year:

  • PyTorch 2.0 and TorchVision 0.15 included a complete revamp of the augmentation pipelines that now natively support object detection, semantic and instance segmentation, and more through the same interface. 
  • The CPython team released Python 3.12 with better and more flexible f-strings (PEP 701). 
  • PyTorch-Ignite’s Code-Generator web application enhanced the code templates and added a Nebari integration. 
  • The Data APIs consortium launched and created an initial RFD for the DataFrame API standard, for cross-dataframe collaboration in the PyData ecosystem. 
  • conda 23.10 adopted conda-libmamba-solver as the new default solver, dramatically reducing solver times for all conda users. 
  • conda-forge created a Streamlit app to stream and serve conda package metadata online. 
  • NumPy’s Python API setup was completely refactored with a single location for all public API symbols, and an adequately hidden private API.
  • Thanks to ongoing funding from the Chan Zuckerberg Initiative (CZI) and the Space Telescope Science Institute (STScI), the Jupyter Accessibility project, which includes several Labs members, continued to raise awareness around accessibility and to make Jupyter tools more accessible.

Quansight and Quansight Labs’ leadership positions in open source.

The image shows a horizontal bar chart displaying the number of maintainers (violet bars on the left) and contributors (purple bars on the right) for several open-source Python data science projects. The projects include PyTorch with around 10 maintainers and 6 contributors, SciPy with around 9 maintainers and 5 contributors, NumPy with around 10 maintainers and 13 contributors, Jupyter with around 7 maintainers and 9 contributors, Conda-Forge with around 6 maintainers and 11 contributors, pandas with around 12 maintainers and 10 contributors, and Scikit Learn with 1 maintainer and 4 contributors.

Quansight Translates Open Source Expertise Into Client Value

Together we’re advancing the open source software underlying over 90% of today’s critical technology.

Quansight’s desire to sustain and advance open source software (as illustrated by our sponsorship of Quansight Labs) directly reinforces our ability to deliver transformative data solutions to clients across industries. The first-hand knowledge our team gains by contributing to very core libraries like NumPy, Pandas, Jupyter, and PyTorch provides an unparalleled level of expertise for our clients.

We intimately understand the architectures, performance optimization strategies, and inner workings of the foundational tools. This makes Quansight uniquely equipped to build customized, high-performance solutions using these projects but tailored to each client’s specific needs.

For example:

  • Quansight has the largest team of PyTorch maintainers outside of Meta and we continue to partner with Meta and the larger community to enhance PyTorch. This includes enhancements to the torch.func module of the PyTorch codebase for last year’s PyTorch 2.0 release.
  • Quansight and the University of Southern California joined forces last year to make Pandas available for use in the paleogeosciences by implementing a community-requested feature dating back to 2014.
  • After seeing certain problems surface time and again on client projects, Quansight’s engineers created three open source projects, two of which are now community-led, to serve both the community and our clients:
    • Nebari, a customizable and open source, enterprise data science and MLOps platform.
    • conda-store, an open source tool created to better manage data science environments for teams.
    • Ragna, an open source RAG-based AI orchestration framework designed to scale from research to production.

Quansight’s open source expertise has empowered similar innovative implementations for clients across domains — from optimized cloud deployments to cutting-edge AI/ML applications. And crucially, the challenges we encounter working with industry leaders flow back as contributions to strengthen the open source projects themselves.

This ability to turn open source knowledge into competitive solutions is a key differentiator for Quansight as an enterprise service provider. We don’t just utilize open source tools, we intimately steer their development.

Quansight’s expertise across the Python data stack as seen in

Generative AI: Made Possible by a Mountain of Open Source.

Open source contributions lead to better software, enabling more robust solutions for clients, whose needs then drive further innovation in open source. Quansight Labs plays a pivotal role in advancing Quansight’s overarching mission to drive business success with open source solutions.

If you’re looking to leverage the power of open source for your business, or you’re interested in talking further about open source sustainability, we want to hear from you.   

And be sure to check out the Quansight Labs Annual Report 2023 for an overview of Labs’ mission and impact.

Share the Post:

More articles from our Library