Windows Noob friendly database solution

For now the database is just 2M but it can grow in future. Since my company is a subsidiary and if Parent company decides to use us more frequently then the DNC (Do not contact) database will increase substantially. The chances of this is high as our small team of 10 people is famous in just 6 months for delivering accurate leads on time.

There is another usecase that I can think of is making a database of scraped data so when a new request comes we can pull all the matching rows for specific account. Search by Company Name or by website or Linkedin company ID etc.... so again its similar to vlookup. Right now we don't need to analyze the data but pull the records and keep merging new lists to existing database
 
kartikoli said:
For now the database is just 2M but it can grow in future. Since my company is a subsidiary and if Parent company decides to use us more frequently then the DNC (Do not contact) database will increase substantially. The chances of this is high as our small team of 10 people is famous in just 6 months for delivering accurate leads on time.

There is another usecase that I can think of is making a database of scraped data so when a new request comes we can pull all the matching rows for specific account. Search by Company Name or by website or Linkedin company ID etc.... so again its similar to vlookup. Right now we don't need to analyze the data but pull the records and keep merging new lists to existing database
Click to expand...
I would still suggest starting with SQLite. The two big advantages for now would be:
1) It's very forgiving as far as RDBMSes go. Backup your sqlite files, and you'd be good to go.
2) As it's rather basic (but still extremely powerful, maybe more so than Excel if you introduce any sort of automation), once you do need to go bigger, your skills will pretty much work from the get-go for other RDBMS.


There's also another way. If you're working directly from CSVs, you might want to consider something like DuckDB. If you want to expand in future, there are Postgres and many other popular options.

As per the “Big Data” side of things, I doubt it'll be coming to your picture anytime soon (I say this because people seem to think x TB of data means Big Data, which is… not true).
 
KAKAN said:
I would still suggest starting with SQLite. The two big advantages for now would be:
1) It's very forgiving as far as RDBMSes go. Backup your sqlite files, and you'd be good to go.
2) As it's rather basic (but still extremely powerful, maybe more so than Excel if you introduce any sort of automation), once you do need to go bigger, your skills will pretty much work from the get-go for other RDBMS.


There's also another way. If you're working directly from CSVs, you might want to consider something like DuckDB. If you want to expand in future, there are Postgres and many other popular options.

As per the “Big Data” side of things, I doubt it'll be coming to your picture anytime soon (I say this because people seem to think x TB of data means Big Data, which is… not true).
Click to expand...
Thanks for the suggestion though I've postponed any plan to switch to DB, right now I am using Python to vlookup excel files and find matching rows. This works great and saves my personal time. Though I do have another plan that includes creating a DB for internal purpose of my website. I've shared the details in this thread here https://techenclave.com/threads/need-some-info-regarding-website-development.215027/
My friend from old company is helping in building a database so everything is on pause at the moment. He will give me an idea about the cost when DB is created which makes me bit anxious because I will just have to wait that long.
 
Back
Top