I’ve been grinding through raw transaction logs lately and wondering if I’m making things too hard. Are you guys sticking with SQL for the heavy lifting or just pulling everything straight into Pandas to manipulate the data?
I personally use SQL for most of my on-chain data analysis since it's straightforward and easy to work with massive datasets, but I've also been experimenting with Pandas to see if it can handle more complex queries. One of the things that's been holding me back from fully switching is the steep learning curve of some of the Pandas functions, but once you get the hang of it, it's super powerful. Has anyone else made the switch or do a hybrid approach like I do?
Honestly I've been using Pandas for my on-chain analysis, it's so much easier to work with and visualize the data compared to SQL. I'm sure SQL has its advantages when dealing with big datasets, but for most projects I've worked on Pandas has been a lifesaver. Does anyone have experience using SQL for more complex on-chain analysis?
Personal opinion, but I've found Pandas way more convenient for on-chain analysis, especially when working with larger datasets. The syntax is way more intuitive for me, and libraries like pandas-datareader make it a breeze to fetch and manipulate data. Plus, it integrates well with other popular data analysis tools.
I'm using Pandas for most of my on-chain analysis, it's just so much easier to work with than SQL when you're dealing with large datasets. That being said, I do know some guys who swear by SQL for their analytics, guess it just depends on what you're trying to accomplish. For me, Pandas has been a game changer for visualizing and manipulating blockchain data.
SQL all the way for the heavy lifting, though I keep Pandas handy for quick local scripts. No point loading gigabytes of data into RAM when you can just query Dune or BigQuery directly.