DB - SQL, NoSQL

Based on project requirements, I have worked with SQL (relational) and NoSQL (non-relational) databases.

In Python, I frequently used Pandas to load data from databases into DataFrames for processing and export.

My experience in Java includes using APIs like JDBC and ORMs such as Hibernate for database access and management.

In C++, I have worked with the c-tree Plus database library, which supports custom index structures and can be configured to operate in either SQL or NoSQL mode.

Below is a description of how c-tree Plus was used in one of my projects.

All database-related activities were my responsibility, covering both data management and code. We have used the c-tree Plus library as an embedded db engine in the application process address space.

To simplify and standardize database access, I developed an application-specific Object-to-Datastore Mapping library-essentially a wrapper around c-tree. This wrapper was used by the entire development team for all database calls.

Later, I added SQL support to the library, enabling integration with Sybase as an SQL database. The transition from c-tree to Sybase was smooth, and developers had the flexibility to choose between an embedded database and a Sybase server, depending on the needs of each project.

SQL

Many of the projects I worked on required handling SQL at various levels of complexity.

In one project, I implemented a conversion from Oracle and PL/SQL to Python, Pandas, and MySQL. While Pandas was used, a significant portion of the work was implemented in SQL scripts.

In another project involving PostgreSQL, I wrote a database schema conversion tool and implemented a synchronization module between the application cache and the database, and wrote triggers and stored procedures in PL/pgSQL, C, and Python. It's a very nice feature of PostgreSQL to have different languages that can be used for stored procedures. In this project, replacing PL/pgSQL procedures with Python-based logic resulted in simpler code.

NoSQL

My experience with NoSQL databases includes object-oriented (OO) databases, graph databases, and in-memory key–value stores.

OO database was used for a Quartz based project.

In a project involving a graph database (Neo4j), I was assigned a ticket to resolve a severe performance issue. After some investigation, I found that depending on the data, one Neo4j query was taking nearly an hour to complete, while other queries executed in seconds. It appeared to be a Neo4j issue likely fixed in later releases, as we were using an outdated version. I found a workaround that reduced the query time from about an hour to several seconds. Neo4j’s comprehensive and well-written documentation proved to be very helpful.

Below are some of the databases I have worked with:

PostgreSQL, Oracle, MySQL, SQLite, Sandra, Neo4j, SYBASE, SQL Server, Redis