Pandas supports Copy-on-Write, an optimization technique that helps improve memory use, particularly when working with large datasets.
  • Nomecks@lemmy.caEnglish
    3·
    1 month ago

    Is there a benefit to doing CoW with Pandas vs. offloading it to the storage? Practically all modern storage systems support CoW snaps. The pattern I’m used to (Infra, not big data) is to leverage storage APIs to offload storage operations from client systems.