← Back to context

Comment by chicob

6 years ago

I once subscribed to a farming management program, developed by a french company that was launching to the cloud market.

Apparently they had bought several other smaller companies, integrated their software into their own program when useful, built huge servers for their cloud service that they also rent.

I knew the program was good - I was familiar with previous versions - so it was kind of a no brainer. The problem was the new cloud thing.

I am in favor of economic rational practices in agriculture (in addition to the normal functioning of markets), so that crops are chosen wisely between farmers and prices do not plunge because of all-or-nothing decisions, and sharing information between farmers is essential.

But data harvesting (no pun intended) of farming practices to the purpose of informing large companies and futures markets brokers of their possibilities can work against the interests of farmers and consumers. The way large corporations treat small farmers is ruthless and they negotiate no terms.

So if my and other farmers' data was being uploaded to the cloud without possibility of local storage, who had access to it? Every crop, production, pesticide, invoice, price was going there!

They wouldn't sell it, they told me. It is stored with e2ee, they assured me. If I ever forgot the password, data would be irrecoverable, they said.

But there was also the problem of internet connection issues, because farms are remote. Would it ever work?

The first thing I noticed after installing the program was that there no important message was shown that informed the user of what would happen if the password was lost. I typed in some new data, saved and exited. Then I started the program and reset the password with ease. There's no possibility of 3FA, either.

So I told them I could not use the program because of my poor internet connection, which was true. They installed a local version that was temporarily available.

But they took so long doing this that I developed my own modular managing system, based in - yes, that's right - spreadsheets. It fits my needs, it is completely portable and it works.

The formulas are getting cryptic, and there are still some things I have to do manually (like sorting lists or generating pdfs), but the files access each other and it works. I have no time to code anything and I don't think I have sufficient knowledge of database software (I am trying to find time to learn MySQL).

Maybe some day I'll make some GUI that handles all of this, in such a way that if everything else fails, I can still access and edit files manually.

As a thought, since you're just getting into database stuff as time permits, it's probably better to start out with something like SQLite and/or DB Browser for SQLite, if you prefer the GUI way of doing things.

https://sqlitebrowser.org

https://a-gentle-introduction-to-sql.readthedocs.io/en/lates...

SQLite is a bunch easier to get up and running than MySQL, and the data is all in one file you can copy around. So, easy to backup. :D

If you do eventually hit it's limits, it's big brother is PostgreSQL, which is a really powerful database system. :)

  • I second that. SQlite is great. I had millions of lines in there and did not hit limits that were blocking me from doing analysis work in python based on the data in SQlite.

    So parent should probably be fine with a solution like this.

    Portable, local and without the server configuration hassel.

  • If you use LibreOffice then you can make a spreadsheet into a "database source" and query it with SQL. That seems like a good interim, you can move sheets and make them external tables as you grow in confidence/knowledge.

    At least you could 10 years ago, very easily (it was OOo.org I was using then I think) - I made a simple customer record system just 4 or 5 tables but the address "table" was a kind-of view of the spreadsheet for email-merge.

    Presumably Excel and others do this too.

    • I do use LibreOffice. There seems to be some way of using SQL to access Base data, but I still didn't explore that.

  • Thanks. I'll give it a try. I chose MySQL because I knew there were Python libraries I could use.

Thank you. Thank you for being self sufficient. Thank you for learning what you need to help yourself.

If you ever need a helping hand. Maybe with visualisation or analysis stuff shoot me a note.