Here comes the story of SMD (which stands for Specialized Media Dashboard), an international project that comprises an open-source media monitoring system based on Google News. The specialized media in question encompasses the spheres of journalism, law enforcement, and climate change. For instance, their most recent project under development is an extensive database-centered system dedicated to the climate change crisis news, raising awareness of the critical global warming issues—the issues whose relevance is expected to grow over the next ten years. This system will expose environmental crimes and thus take part in the world’s efforts to prevent global climate change.
The challenges
Yet our today’s case dates back to early 2020, when the world witnessed the beginning of the COVID-19 pandemic. It was only natural for Specialized Media Dashboard to focus their coverage on this issue, aggregating, analyzing, and processing related news articles in vast numbers. That’s a huge amount of data, and it caused a number of gradually growing challenges.
- The sheer number of incoming information was increasing up to the point when it became just too large to handle
- This caused the need to populate SMD’s MariaDB databases with all that data automatically
- The SMD team also needed to sort that data by categories, sub-categories, and keywords
That’s where the trouble was. SMD’s then-current database tools were simply unable to handle those challenges. They virtually didn’t have the means and capacities to process that much data. But a little bit of research led SMD to dbForge Studio for MySQL, our multifunctional IDE that proved to be capable of manipulating and processing millions of records without any impact on performance. It was exactly what the software engineers of SMD were looking for.
Daniel K., the Climate Change Crisis Project Coordinator, noted: “Thanks to its administration tools, SMD engineers were able to manipulate and process millions of records in a faster and more efficient way. We became 50% more productive.”
The results
Besides the abovementioned performance, dbForge Studio delivered tools for the full automation of news processing. Articles could be easily sorted, automatically sent to Google documents, and then exported to MariaDB databases in the blink of an eye. The entire process was conveniently streamlined, much to the relief of the team.
It is also worth noting that SMD became this year’s finalist for a prestigious MariaDB award in the Database Transformation of the Year category. Daniel noted that this accomplishment would not have been possible without the contribution of our tools. We are thankful for this, and we are happy to be part of a noble cause.
The key features of the Studio noted by the SMD team
Flexible import and export of data
Data export and import between different sources and MySQL/MariaDB databases have never been easier with dbForge Studio at hand. The supported formats include HTML, TXT, XLS, XLSX, MDB, RTF, PDF, JSON, XML, CSV, ODBC, DBF, SQL, and Google Sheets. For each of these formats, dbForge Studio provides a rich set of flexible settings, options, and, of course, CLI-powered automation capabilities that helped the SMD team streamline their routine operations.
Backup and recovery
When working with such huge volumes of data, it is also vital to make sure that accidental data loss is not an issue. That’s why Daniel pinpointed the reliable backup functionality of the Studio. Like with the previous feature, automation does a great job here and relieves the team of tiresome manual operations, considering how much data they have to work with.
Database comparison and synchronization
Lastly, Daniel’s team singled out the handy built-in tools for the comparison and synchronization of database schemas/objects and actual table data, which can be easily carried out in the Studio. Associated features include quick comparison of a source and a target, convenient highlighting of differences for analysis, and instant generation of synchronization scripts that can be either run immediately or scheduled for execution.
