← Back to context

Comment by tonymet

7 hours ago

i had been thinking about this service for a long time, especially something supporting transforms and indexing for backups. great job spinning it up.

2 comments

tonymet

Reply
coreylane  6 hours ago

Thanks 1. are you thinking of something like aws data firehose transform feature? where pandas or something can run inline? https://docs.aws.amazon.com/firehose/latest/dev/data-transfo...

2. do you have an example of what indexed backups would look like? Im thinking of macos time machine, where each backup only contains deltas from the last backup. Or am I completely off?

  • tonymet  3 hours ago

    For transforms, the concept would be user friendly processing, like downcoding video & photos, compressing PDFs & text files, filtering out temporary or wasteful files . Something like AirTable for backups with a gui workflow editor with common processing jobs for backups.

    For indexing, full text indexing of backups to allow for record retrieval based on keyword or date. E.g. “images in Los Angeles before 2010” or “tax records from 2015”. If possible, low resolution thumbnails of the backups to make retrieval easier.

    I think #1 (transforms) would be more generally useful for cross cloud applications, and #2 is more catered toward backups