Cloud database services are easy to love at the start. You sign up, provision a database instance in minutes, and pay only for what you use. There's no hardware to buy, no data center to maintain, and no upfront capital commitment. For early-stage projects and small teams, this model is genuinely hard to beat. But as workloads mature and data volumes grow, the financial picture often becomes more complicated - and more expensive - than the initial simplicity suggested.

The Sticker Price Is Just the Beginning

Cloud providers price their database services in ways that make small-scale usage look attractively cheap but cause costs to compound quickly as usage scales. The base instance cost is only the starting point. Storage is billed separately, and in most managed database services, storage pricing is meaningfully higher than raw object storage costs. That's because you're paying for managed, redundant, high-performance disk, not just bytes on a drive. Backup storage is often billed on top of that, and retaining backups for compliance purposes can add up to a surprisingly large monthly line item.

Compute costs follow a similar pattern. The instance sizes that handle light development traffic become inadequate as production workloads grow, and stepping up to the next tier often means a significant jump in hourly cost. Reserved instance pricing can reduce this, but it requires committing to one or three years of usage upfront, which reintroduces a form of capital commitment that cloud was supposed to eliminate.

Egress Fees: The Cost Nobody Talks About Enough

One of the most underappreciated costs in cloud database operations is data egress, i.e., what you pay to move data out of the cloud provider's network. Ingress (data coming in) is typically free. Egress (data going out) is not, and the rates can be substantial when you're regularly transferring large result sets to analytics platforms, downstream applications, or on-premise systems. Organizations that run hybrid architectures - with some systems in the cloud and others on-prem - often discover that inter-environment data movement is quietly one of their larger cloud expenses.

This is worth thinking about carefully during architecture planning, because the impact isn't always obvious until you're already paying for it. A reporting pipeline that runs daily queries and exports results to an on-prem data warehouse might look cheap in compute terms but become expensive once egress is factored in.

Operational Costs Don't Disappear (They Transform)

A common argument for cloud database services is that they eliminate operational overhead: no DBAs patching servers, no hardware failures to diagnose, no capacity planning to worry about. This is partially true, but it replaces one set of operational concerns with another. Someone still needs to manage database configurations, monitor performance, tune queries, manage credentials and access controls, and respond to incidents. What changes is the nature of the work, not the need for skilled people to do it.

Tooling costs also tend to accumulate alongside cloud database spending. Monitoring, observability, backup management, and security scanning are all areas where organizations commonly bolt on third-party services - each with its own subscription fee - to fill gaps in what the cloud provider offers natively.

When On-Prem Makes More Financial Sense

The economics of on-premise infrastructure tend to favor organizations that have steady, predictable workloads rather than spiky or seasonal demand. If you're running database servers at consistently high utilization, say, above 60 to 70 percent, the cost per unit of compute on owned hardware is typically lower than the equivalent cloud instance cost over a three-to-five year hardware lifecycle. The crossover point varies by organization, but it's often reached earlier than people expect.

Organizations that have already invested in a data center, network infrastructure, and an in-house IT team to manage it are in a particularly strong position to benefit from on-prem database hosting. The marginal cost of adding database capacity to existing infrastructure is much lower than it would be for an organization starting from scratch. For these teams, the cloud's selling point of "no infrastructure to manage" is less compelling, because the infrastructure already exists and the people to run it are already on staff.

Data volume is another factor. Very large databases (multi-terabyte or petabyte-scale) can generate storage and egress costs in the cloud that dwarf the cost of equivalent on-prem storage hardware. At sufficient scale, buying and managing your own storage is simply cheaper, even accounting for the overhead of doing so.

Reducing Complexity and Regaining Cost Control with Navicat On-Prem Server 3.1

One of the less obvious contributors to rising database costs in cloud environments is the fragmentation of tooling and access management. As teams grow, it's common to layer multiple services for user management, collaboration, monitoring, and query workflows, each adding incremental cost and operational complexity. This is where solutions like Navicat On-Prem Server 3.1 fit naturally into an on-premise or hybrid strategy.

By centralizing database access, user permissions, and collaborative workflows within your own infrastructure, Navicat On-Prem Server 3.1 helps reduce reliance on multiple cloud-based tools and subscriptions. Teams can manage queries, share connections, and control access from a single platform without incurring ongoing per-user or usage-based cloud fees. This aligns particularly well with organizations already operating on-prem systems, where predictability and cost containment are key priorities.

There is also a data locality advantage. Keeping database management and access layers within the same environment as the data itself minimizes unnecessary data movement, which in turn helps avoid the egress charges that often accumulate in cloud-heavy architectures. Over time, these incremental savings can be meaningful, especially for data-intensive workloads.

In this sense, tools like Navicat On-Prem Server 3.1 are not just operational conveniences; they are part of a broader strategy to simplify architecture, consolidate tooling, and bring database-related costs back under direct organizational control.

Conclusion

Neither hosting model is universally cheaper. The right answer depends on your workload characteristics, your existing infrastructure, your team's capabilities, and your organization's financial preferences around capital versus operating expenditure. The important thing is to make that comparison honestly, with all the costs on the table, rather than letting the initial simplicity of cloud pricing obscure what you'll actually be paying once your system is running at scale.

For most of its history, writing SQL has been a largely manual craft. A database administrator or developer would pull up a query editor, recall the relevant table names and column definitions from memory or (more likely!) a schema diagram, and construct statements piece by piece. Syntax errors were caught at execution time. Optimization was a separate, deliberate step. Now, AI-powered code completion is beginning to change that workflow in meaningful ways - not by replacing the human (at least, not yet!), but by compressing the distance between intent and working query.

What AI Code Completion Actually Does

Traditional code completion, i.e., the kind that has been in database IDEs for years, works by pattern-matching against known SQL syntax and object names in the connected schema. It can suggest a table name after you type FROM, or complete a column name once it recognizes the context. Useful, but fundamentally mechanical.

AI-powered completion goes further. Rather than just predicting the next token based on syntax rules, it understands intent. You can describe what you want in plain language, for example, "find all customers who placed more than three orders in the last 90 days", and the AI can generate a complete, structurally sound SQL statement. It can also suggest how to rewrite a subquery as a JOIN, flag a missing index condition, or explain why a particular query might perform poorly at scale. The difference is less about autocomplete and more about having a knowledgeable collaborator available at the point of writing.

The Practical Impact on DBA Workflows

The most immediate benefit of AI-powered completion is speed. Routine queries such as aggregations, filtered selects, and common JOIN patterns that would take a few minutes to write carefully can often be scaffolded in seconds, leaving the DBA to focus on reviewing and refining rather than constructing from scratch. For less experienced team members, this is particularly valuable: AI suggestions provide a working starting point and implicitly model good query structure, which accelerates learning in a way that blank-editor writing does not.

There are also gains in consistency. When multiple developers are working across the same schema, AI tools can help enforce consistent patterns for things like date filtering, NULL handling, and aggregation logic, hence reducing the subtle variability that tends to creep into large SQL codebases over time.

That said, AI-generated SQL still requires careful human review. The output is only as good as the context provided, and models can confidently produce queries that are syntactically valid but semantically wrong - joining on the wrong key, filtering on the wrong column, or missing a critical business rule that the AI had no way of knowing. The DBA's judgment remains indispensable; AI assistance changes where that judgment is applied, not whether it's needed.

AI Features in Navicat On-Prem Server 3.1

Navicat On-Prem Server 3.1, released in February 2026, brought AI Assistant and Ask AI into the on-premise collaboration platform for the first time - making these capabilities available to teams who manage their database infrastructure entirely within their own network.

The AI Assistant provides a conversational interface directly within the platform where users can ask questions and receive immediate answers. This is particularly useful for query writing and explanation tasks: a team member can describe what they're trying to retrieve, ask the assistant to explain an unfamiliar query written by a colleague, or get guidance on SQL syntax without leaving the tool they're already working in.

Ask AI is oriented more toward specific, action-driven tasks in the query editor. Users can invoke it to explain, optimize, format, or convert SQL queries, covering some of the most common tasks that slow down query development. Frequently used actions can be pinned for quick access, which makes the feature practical for day-to-day use rather than something you have to dig for when you need it.

Conclusion

AI code completion isn't replacing the DBA, it's changing the shape of the job. The cognitive load shifts away from syntax recall and boilerplate construction toward higher-order tasks: validating AI output, making architectural decisions, and applying the business context that no model can infer on its own. For teams willing to adapt their workflows thoughtfully, these tools represent a genuine productivity gain. The challenge, as with most AI tooling, is learning where to trust the output and where to intervene - and that judgment, for now, remains entirely human.

Every database holds data that some people only need to view, some need to modify, and others should never touch at all. Role-Based Access Control - commonly referred to as RBAC - is the framework that makes that distinction enforceable. When it's implemented well, it reduces security risk, simplifies auditing, and makes it far easier to manage access as teams grow and change. When it's implemented poorly, it tends to collapse into either over-permissioning (everyone can do everything) or under-permissioning (nobody can do what they need to). Getting it right requires more than just knowing the theory.

What RBAC Actually Means in a Database Context

At its core, RBAC is the practice of assigning permissions to roles rather than directly to individual users. A user is then granted access by being assigned to one or more roles. This indirection is what makes the system scalable: when a job function changes, you update the role once rather than hunting down every individual account that performs that function.

In a database environment, roles typically map to actions like reading data, writing or modifying data, managing schema objects (creating or dropping tables, indexes, and so on), and administering users and permissions themselves. Most production database systems, like MySQL, PostgreSQL, SQL Server, and Oracle, have native support for role-based privilege management, though the implementation details vary considerably between them.

The Principle of Least Privilege

The single most important design principle behind any sound RBAC implementation is least privilege: every user and every role should have the minimum level of access necessary to perform their intended function, and nothing more. This sounds straightforward but is frequently violated in practice, often for convenience. A developer who needs read access to a production database to debug an issue gets granted full read-write access because it's faster to set up. A contractor who needs access to one schema gets access to the entire server. Over time, these shortcuts accumulate into a permissions structure that nobody fully understands.

Least privilege also applies horizontally, not just vertically. A role that needs access to one database shouldn't have it granted at the server level. A role that needs to read from three tables shouldn't have SELECT privileges on the entire schema. Precision matters, both for security and for auditability.

Designing Roles Before Assigning Them

A common mistake is to treat RBAC as something you configure reactively - adding permissions when someone asks for access, removing them when something goes wrong. The more reliable approach is to design your role taxonomy upfront, based on the actual job functions that interact with your databases.

Start by identifying the distinct categories of users: read-only analysts, application service accounts, developers, DBAs, security auditors, and so on. For each category, define exactly what operations they need to perform and on which objects. Then model your roles to match those categories, keeping roles focused and non-overlapping where possible. A user who performs multiple functions can be assigned multiple roles, but each role should be coherent on its own.

It's also worth distinguishing between roles that exist at the database engine level (the privileges assigned within MySQL, PostgreSQL, and so on) and roles that exist at the tooling or collaboration layer, where teams manage shared objects like queries, connection configs, and data models. Both layers need governance.

Managing Access in Navicat On-Prem Server

For teams using Navicat On-Prem Server as their database collaboration platform, access control is managed at the project level through a straightforward three-tier role system. When adding a member to a project, administrators assign one of three access rights that determine what that member can do within the project:

Can Manage and Edit is the highest level of access. Members with this right can read and interact with all objects in the project, create and modify objects, manage project membership (adding or removing other members and adjusting their roles), and rename the project itself. This right is appropriate for project leads, senior DBAs, or anyone who needs administrative control over the collaboration workspace.

Can Edit grants full read and write access to project objects. Members can view and modify shared content, but stops short of membership management and project renaming. This is well-suited to active contributors who need to create and update queries, connection settings, or other shared resources, but who shouldn't have authority over the project's structure or membership.

Can View is a read-only role. Members can access and view objects within the project but cannot make changes to any of them. This is the appropriate choice for stakeholders, auditors, or team members who need visibility into shared resources without the ability to alter them.

This model maps cleanly onto the principle of least privilege: access is scoped specifically to collaboration objects within the platform, and the three tiers cover the most common real-world access patterns without creating unnecessary complexity. It also complements, rather than replaces, the underlying database-level permissions managed within individual database engines; the two layers of access control work together.

Keeping Access Control Maintainable Over Time

RBAC implementations tend to drift. People change roles, projects end, contractors leave, and permissions that were set up temporarily become permanent through neglect. Building in a regular review cadence (quarterly is common) helps keep your permission structure clean. Automated tooling that reports on unused roles, dormant accounts, or privilege escalations can surface problems before they become incidents.

Documentation matters too. When roles are well-documented with clear statements of purpose, who should hold them, and what they grant access to, it becomes much easier for new administrators to maintain the system correctly and for auditors to verify it. An RBAC setup that only one person fully understands is a fragile one.

Conclusion

Role-based access control isn't a configuration you set once and forget. It's an ongoing practice that reflects your organization's structure, security posture, and operational needs. The core principles - least privilege, role-based rather than user-based assignment, and regular review - apply whether you're managing privileges in a database engine directly or governing access to a shared collaboration platform like Navicat On-Prem Server.

When it comes to hosting your databases and the tools that manage them, the choice between on-premise and cloud-based infrastructure is rarely as simple as it looks. Both models have matured considerably over the past decade, and the right answer almost always depends on the specific circumstances of your organization, as opposed to any universal rule of thumb.

What "On-Prem" and "Cloud" Actually Mean

On-premise (on-prem) database hosting means your databases and management infrastructure run on servers you own and physically control, typically within your own data center or office network. The cloud, by contrast, means delegating that infrastructure to a third-party provider - AWS, Azure, Google Cloud, and so on - who hosts and maintains the underlying hardware on your behalf.

A third option, the hybrid model, sits between them: some data and workloads remain on-prem while others move to the cloud. This is increasingly common in large enterprises that have legacy systems they can't easily migrate.

The Case for Cloud Hosting

Cloud database hosting has surged in popularity for good reasons. It eliminates the capital expenditure of buying servers, reduces the operational burden of managing hardware, and makes it trivially easy to scale up or down based on demand. For startups, small teams, or projects with variable workloads, the cloud's pay-as-you-go model is genuinely compelling.

The cloud is also attractive for distributed teams. If your engineers, DBAs, and analysts are spread across different cities or time zones, having your database infrastructure in the cloud makes collaboration simpler and doesn't require VPN tunneling or complex firewall rules to give everyone access.

The Case for On-Prem Hosting

The cloud isn't the right fit for everyone. Organizations in regulated industries such as healthcare, finance, government, and legal often operate under compliance requirements (HIPAA, GDPR, PCI-DSS, SOX) that impose strict controls over where data physically resides and who can access it. For these organizations, keeping databases on-prem isn't a preference; it's often a legal or contractual obligation.

Beyond compliance, on-prem hosting gives you complete control over your security posture, network configuration, and upgrade schedules. You're not subject to a provider's maintenance windows, pricing changes, or service outages. For organizations with steady, predictable workloads and an in-house IT team to manage infrastructure, on-prem can also be significantly more cost-effective in the long run than paying ongoing cloud subscription fees.

Tools That Bridge the Gap

One of the more interesting developments in the database tooling space is the emergence of products designed specifically to give teams the collaboration benefits of the cloud while keeping data entirely on-premise. Navicat On-Prem Server is a good example of this philosophy in practice.

Rather than forcing a choice between collaboration and data sovereignty, it lets organizations host their own private server that all Navicat desktop clients can connect to for real-time team collaboration. Team members can share connection settings, queries, code snippets, data models, and BI workspaces through a centralized hub they control entirely, without any data ever leaving their own network.

The most recent release, version 3.1 (February 2026), adds AI Assistant integration to the platform, including "Ask AI" features directly within the server environment. This brings AI-assisted query writing and code generation into an on-prem context - an important step for organizations that want the productivity benefits that AI tooling offers. The platform already supported MySQL, MariaDB, and PostgreSQL (including Fujitsu Enterprise Postgres) since version 3.0, along with an enhanced query editor featuring code completion, folding, and SQL beautification.

Key Questions to Guide Your Decision

If you're evaluating which model suits your organization, a few questions tend to cut through the noise quickly. Does your industry have data residency requirements that restrict where your data can be stored? If yes, on-prem or a private cloud is likely non-negotiable. Do you have the internal IT staff to manage and maintain database servers? If no, managed cloud services may reduce operational risk. Is your workload predictable or highly variable? Unpredictable, spiky workloads generally favor cloud elasticity, while steady workloads favor on-prem economics. And finally, how important is team collaboration across distributed locations? If real-time sharing is critical, make sure any on-prem solution you choose - like Navicat On-Prem Server - is built to support it natively.

Ultimately, the right answer isn't about which model is objectively better. It's about which one aligns with your compliance obligations, your team's capabilities, and your organization's tolerance for infrastructure risk and cost.

Navicat's latest On-Prem Server (3.1) is bringing AI assistance to database management in a big way. In fact, two of its three new features feature AI: there's a general purpose AI Assistant as well as a more specialized Ask AI tool aimed at SQL development. Both of these rely on APIs of popular AI models. In today's blog article, we'll learn how easy it is to get started with AI Assistants so that your team can benefit from the power of AI guidance.

A Brief Introduction to AI APIs

AI APIs (Application Programming Interfaces) are services that allow developers to access AI capabilities over the internet without having to build or host the underlying AI models themselves. Instead of training your own large language model - which requires enormous computing resources and expertise - you simply send a request to an AI API and receive an intelligent response back in seconds.

The use cases for AI APIs are remarkably broad, which is part of why they have become so popular. They power natural language processing tasks like answering questions, summarizing documents, translating languages, and analyzing sentiment. In software development, they enable code assistance features that can generate, explain, debug, and optimize code - exactly like Navicat's Ask AI tool does for SQL. Beyond development, AI APIs are widely used for content generation, data extraction, chatbots, virtual assistants, and image recognition.

This is why configuring the AI Assistant in Navicat involves selecting and connecting to an AI API provider, essentially giving the software its "brain." So let's move on to that now!

Adding an AI Assistant

All of the details pertaining to AI Assistants are located on the Database Management Settings screen. It's available by clicking the drop-down that is associated with your Profile Name:

On the Database Management Settings screen, click the AI button at the top of the dialog to display all of the AI Assistant details. At first it will be mostly empty because we have not yet selected and AI Model. To do that, click the plus (+) icon at the bottom of the AI Assistant list:

Doing so will open a menu containing all of the available AI model providers. All of the most popular providers are supported, including Anthropic Claude, Google Gemini, DeepSeek, Grok, and more.

Once you've selected an AI model provider, a number of fields will appear to the right of the AI Assistant list. These will vary depending on the provider you choose. For example, Claude supports a max_tokens parameter which we see on the screen as "Max Tokens". The right number for Max Tokens depends on what you're using Claude for, but here are some guidelines:

• Simple confirmations/classifications: 256-512 tokens
• Chat/Q&A: 1024-2048 tokens is usually plenty
• Code generation: 4096-8192 tokens for complex functions
• Writing/content creation: 4096-8192 tokens

Important things to know:

• max_tokens is the maximum Claude can generate - it will stop when it's finished, even if it hasn't reached the limit
• 1 token ≈ 4 characters or roughly 0.75 words in English, so 1024 tokens ≈ 750-800 words
• The maximum varies by model - Claude models support up to 8192 output tokens (though the total context window is much larger)
• You're only charged for tokens actually used, not the max you specify

If you want to limit the length of responses, start with 4096 tokens - this gives Claude plenty of room for detailed responses without being unnecessarily high. If you find responses are getting cut off, increase it. If you know you only need brief answers, you can lower it to save a bit on costs. Remember that setting it higher doesn't cost you more unless Claude actually uses those tokens, so it's often safe to be generous with this parameter.

Obtaining an API Key

Regardless of which AI provider you choose, the API Key is the most crucial piece of information that you'll need to provide. You'll need to obtain an API key from your chosen AI provider. An API key is essentially a unique, auto-generated password that identifies your application and grants it access to the AI service. When Navicat On-Prem Server sends a request to an AI API, it includes this key as part of the request, allowing the provider to verify your identity, track your usage, and apply the appropriate billing charges.

API keys are obtained by registering for an account on your chosen provider's platformsuch as Anthropic's Console at console.anthropic.com or OpenAI's platform at platform.openai.com and are typically generated instantly on demand. Since an API key provides direct access to a paid service, it should be treated like a password: stored securely, never shared publicly, and never embedded in publicly accessible code. Most providers allow you to generate multiple keys for different applications or team members, and to revoke them instantly if they are ever compromised.

Choosing a Model

The most important decision you will need to make - next to the provider - is what model to use. Most AI providers offer several to choose from. Clicking the ellipsis (...) to the right of the Model textbox opens a dialog where you can select a model. Here are the choices for Claude:

It's worth noting that the cost for using different models can vary significantly. Consult the provider's model overview page for exact pricing information. For example, here is the overview page for Claude.

Putting It All Together

Once you've filled in all of the necessary information, you can test the API service by clicking the "Test Connection" button. If all goes well, you should see a "Connection Successful!" message at the top of the screen:

You're now ready to use both On-Prem Server 3.1's AI Assistant and Ask AI tools!

Choosing between SQL and NoSQL databases is one of the most critical architectural decisions you'll make in any project. While the industry hype cycle has swung wildly between championing relational databases and promoting NoSQL as the future, the reality is that each approach serves distinct purposes. Making the right choice requires understanding your specific requirements rather than following trends.

Understanding the Core Differences

SQL databases like MySQL, PostgreSQL, and SQL Server organize data into structured tables with predefined schemas and relationships. They excel at maintaining data integrity through ACID properties, making them ideal for applications where consistency is paramount. Meanwhile, NoSQL databases such as MongoDB and Redis take varied approaches, storing data as documents, key-value pairs, or graphs without rigid schemas. This flexibility allows them to scale horizontally and handle rapidly changing data structures.

When SQL Makes Sense

Traditional relational databases remain the optimal choice when your data has clear relationships and structure. Financial applications, e-commerce platforms with complex transactions, and systems requiring robust reporting capabilities benefit from SQL's powerful join operations and transaction guarantees. If your application needs strong consistency, complex queries across multiple tables, or regulatory compliance with strict data integrity requirements, SQL databases provide proven, reliable solutions.

When NoSQL Shines

NoSQL databases excel in scenarios requiring massive scale, high write throughput, or flexible data models. Real-time analytics platforms, content management systems with diverse data types, IoT applications processing millions of sensor readings, and mobile apps requiring offline sync capabilities often perform better with NoSQL. The ability to evolve your schema without migrations and to distribute data across multiple servers makes NoSQL particularly attractive for rapidly growing applications.

The Hybrid Reality

Many modern applications don't fit neatly into either category. You might use PostgreSQL for transactional data while employing Redis for caching and session management, or combine SQL Server with MongoDB for handling both structured customer records and unstructured product catalogs. This approach - known as "polyglot persistence" - leverages each database type's strengths.

Managing Both SQL and NoSQL with Navicat

Navicat eliminates the complexity of working across different database types. Navicat Premium provides a unified interface for managing both SQL databases including MySQL, PostgreSQL, MariaDB, SQL Server, Oracle, SQLite, and Snowflake, alongside NoSQL systems like MongoDB and Redis, all within a single application. This means developers and database administrators can switch between relational and NoSQL databases without learning multiple management tools.

The platform's visual query builder works seamlessly across different database types, while features like data modeling, synchronization, and backup operate consistently whether you're working with SQL tables or NoSQL collections. Navicat's support for MongoDB includes schema visualization and aggregation pipeline builders, while its Redis integration provides intuitive interfaces for key-value operations. This unified approach proves invaluable when implementing hybrid architectures, allowing teams to design, develop, and maintain complex data ecosystems efficiently.

Making Your Decision

Choose based on your actual requirements, not industry buzz. Consider your data structure, consistency needs, scalability requirements, and team expertise. Remember that you're not locked into a single choice forever. Start with the database that best fits your current needs, and use tools like Navicat to manage complexity as your architecture evolves.

For years, many organizations have relied on simple uptime checks to gauge database health. While knowing your database is running is certainly important, uptime alone tells you almost nothing about performance, efficiency, or the user experience. A database can technically be "up" while delivering painfully slow queries, suffering from resource contention, or teetering on the edge of capacity exhaustion. Modern database monitoring requires a more sophisticated approach that focuses on metrics that actually impact your applications and users.

Query Performance Metrics

The most critical area to monitor is query performance, since queries are where your database directly interacts with your applications. Long-running queries are often the canary in the coal mine for deeper problems. By tracking query execution times, you can identify which specific queries are consuming excessive resources and causing bottlenecks. Equally important is understanding query wait times, which reveal what your queries are waiting for, whether that's disk access, locks, or network resources.

Beyond execution time, examining the top queries by CPU usage helps you identify which operations are most computationally expensive. Similarly, tracking queries by the number of reads and writes they perform can highlight inefficient data access patterns that might benefit from index optimization or query refactoring. These metrics transform abstract performance concerns into concrete, actionable insights.

Resource Utilization and Capacity

While CPU and memory usage might seem like basic metrics, understanding them in context is crucial. CPU utilization patterns tell you whether your database server has adequate processing power for your workload, but more importantly, sustained high CPU usage can indicate missing indexes or poorly optimized queries rather than simply insufficient hardware.

Memory metrics deserve particular attention because databases rely heavily on caching to achieve good performance. The buffer cache hit ratio, which measures the percentage of data requests served from memory rather than disk, should typically exceed 90 percent. When this ratio drops, it indicates that your database is frequently going to disk for data, dramatically slowing performance. Monitoring memory allocation over time also helps with capacity planning, showing you whether your database's memory footprint is growing at a sustainable rate.

Disk I/O metrics complete the resource picture. Tracking disk read and write operations per second, along with average disk response times, helps you understand whether storage is becoming a bottleneck. Network I/O is equally important for understanding how much data is flowing between your database and applications.

Connection and Session Activity

Monitoring active connections and session details provides visibility into how your applications are actually using the database. Tracking current user connections helps you understand your concurrent workload and can alert you to connection pool exhaustion before it causes application failures. Monitoring connection patterns over time also reveals usage trends that inform capacity planning decisions.

Lock monitoring is particularly critical for understanding contention issues. When queries are waiting for locks held by other sessions, users experience delays that simple CPU or memory metrics won't explain. By tracking both the locks currently held and sessions waiting for locks, you can identify problematic transaction patterns or long-running transactions that are blocking other work.

Measuring These Metrics with Navicat Monitor

Navicat Monitor provides an agentless architecture for monitoring MySQL, MariaDB, PostgreSQL, and SQL Server databases, which means you don't need to install software on your database servers themselves. The tool collects metrics at regular intervals and stores them in a repository database for historical analysis and trending.

For query performance monitoring, Navicat Monitor's Long Running Queries chart visualizes top queries based on execution duration, wait types, CPU usage, and read/write operations. This allows you to quickly identify problematic queries and drill down into their execution characteristics. The tool maintains historical data so you can track whether query performance is degrading over time.

Resource monitoring in Navicat Monitor covers the full spectrum of system metrics. It collects CPU load, RAM usage, and various other system resources over SSH or SNMP, giving you visibility into both database-level and operating system-level performance. The interactive dashboard provides real-time and historical graphs showing server load, disk usage, network I/O, and table locks, making it easy to correlate different metrics and identify patterns.

One particularly powerful feature is the custom metrics capability. You can write your own queries to collect performance metrics for specific instances and receive alerts when values exceed defined thresholds. This means you can monitor business-specific indicators or specialized performance characteristics that matter to your particular applications, going well beyond the standard preset metrics.

The alerting system in Navicat Monitor enables proactive management by notifying you when metrics cross configurable thresholds. You can set alerts for any metric, including custom ones, and define both the threshold value and how long it must be exceeded before triggering an alert. Notifications can be delivered via email, SMS, SNMP, or Slack, ensuring your team knows about problems before they impact users. The tool provides detailed alert analysis that includes metric charts, timelines, and historical context to help with root cause analysis.

Beyond the Dashboard: Making Metrics Actionable

Collecting metrics is only the first step. The real value comes from understanding patterns, setting appropriate baselines, and creating actionable alerts. Rather than simply watching dashboards, establish normal ranges for your key metrics based on historical data and workload patterns. This allows you to set intelligent alert thresholds that catch genuine problems without generating false alarms from normal variations.

Consider the relationships between metrics when investigating issues. A spike in disk I/O might correlate with a drop in buffer cache hit ratio and an increase in query execution times. Understanding these connections helps you identify root causes rather than just symptoms. Regular capacity planning reviews using historical trends ensure you can scale proactively before hitting resource constraints.

Moving from simple uptime monitoring to comprehensive performance monitoring will significantly impact how you understand and manage your databases. By focusing on metrics that directly impact application performance and user experience, you can move from reactive fire-fighting to proactive optimization, ensuring your databases deliver consistent, reliable performance.

Every modern application that stores data faces a fundamental challenge: how do you let multiple users work with the same database at the same time without their actions corrupting each other's data? Without proper safeguards, concurrent operations could produce incorrect results, duplicate transactions, or delete crucial information. Database transaction isolation levels exist to solve concurrency issues, giving you a toolkit of different strategies for managing concurrent access. Each isolation level represents a different answer to the question of how much transactions should be aware of and affected by each other's work. As you'll discover in this article, choosing the right isolation level means understanding the trade-offs between data accuracy, system performance, and the types of anomalies you're willing to accept in your application.

What Are Transaction Isolation Levels?

When multiple users access a database simultaneously, transactions can interfere with each other in unexpected ways. Transaction isolation levels determine how much one transaction can see or be affected by changes made by other concurrent transactions. It's helpful to think of isolation levels as different approaches to balancing two competing needs: maintaining data accuracy and allowing multiple people to work with the database at the same time. Higher isolation levels provide stronger guarantees about data consistency but can slow down your system, while lower levels offer better performance at the cost of potential data anomalies.

Read Uncommitted: The Lowest Protection

Read Uncommitted is the most permissive isolation level, where transactions can read data that other transactions have modified but not yet saved permanently. This approach prioritizes speed over accuracy. In this mode, you might encounter dirty reads, where your transaction sees changes that could be rolled back moments later. Imagine checking a bank account balance while someone else is transferring money out of it. You might see the reduced balance even though that transfer could fail and be reversed. Read Uncommitted is rarely appropriate for production systems, though it might be acceptable for generating rough reports where perfect accuracy is less critical than speed.

Read Committed: The Common Default

Read Committed prevents dirty reads by ensuring transactions only see data that has been permanently saved by other transactions. This is the default isolation level for most database systems, striking a reasonable balance between performance and reliability. However, Read Committed still allows non-repeatable reads. If you read the same row twice within your transaction, you might get different values if another transaction modified and committed that data between your reads. This level works well for many everyday applications where you need reliable data but can tolerate some changes happening during your transaction.

Repeatable Read: Maintaining Consistency

Repeatable Read goes further by guaranteeing that if you read a row once in your transaction, you'll get the same values if you read it again, even if other transactions are making changes. The database accomplishes this by holding locks on the data you've read until your transaction completes. This prevents other transactions from modifying that specific data. However, Repeatable Read can still experience phantom reads, where new rows matching your query criteria appear between reads. For instance, if you count all orders over one hundred dollars and then count again, new qualifying orders inserted by other transactions might appear in your second count, changing your result.

Serializable: Maximum Isolation

Serializable represents the strictest isolation level, making transactions behave as if they're running one after another in sequence rather than simultaneously. This level prevents all the anomalies that plague lower isolation levels, including dirty reads, non-repeatable reads, and phantom reads. The database achieves this by acquiring range locks that prevent other transactions from inserting, updating, or deleting data that could affect your queries. While Serializable provides the strongest data consistency guarantees, it significantly reduces how many transactions can run simultaneously, which can impact system performance. This level is essential for critical operations like financial transactions where even small inconsistencies are unacceptable.

Working with Isolation Levels in Navicat

Navicat serves as an indispensable graphical interface for managing your database transactions and isolation levels. When you open a query window in Navicat, you're working directly with your database server, and Navicat provides a convenient way to execute the SQL commands that control isolation levels. You can set the isolation level for your current session by running standard SQL commands in the query editor. For example, in SQL Server you would execute SET TRANSACTION ISOLATION LEVEL READ COMMITTED, while in MySQL you might use SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ. Navicat faithfully sends these commands to your database, allowing you to experiment with different isolation levels and observe their effects.

The real power of using Navicat for understanding isolation levels comes from its ability to open multiple query windows simultaneously. You can set different isolation levels in separate windows, then run transactions in parallel to see how they interact. This hands-on approach helps you understand the practical differences between isolation levels. For instance, you could demonstrate a phantom read by setting one window to Repeatable Read and another to Read Committed, then inserting rows in one window while querying in the other. While Navicat doesn't enforce or manage isolation levels itself, since that's the database server's responsibility, it provides an accessible environment for learning and testing how different isolation configurations affect your data operations.

Conclusion

Transaction isolation levels give you control over how your database handles concurrent access, with each level offering a different balance between data consistency and performance. By understanding the trade-offs between Read Uncommitted, Read Committed, Repeatable Read, and Serializable, you can make informed decisions about which level best serves your application's needs. Whether you're building financial systems that demand perfect accuracy or reporting tools that prioritize speed, choosing the right isolation level is essential for creating reliable, efficient database applications.

When your application needs to talk to a database, it must first establish a connection. This process might seem instantaneous from a user's perspective, but behind the scenes, it involves several time-consuming steps: the database server must authenticate credentials, allocate memory for the connection, and set up communication channels. If your application creates a new connection for every database query and then closes it immediately afterward, you're essentially forcing the system to repeat this expensive setup process hundreds or thousands of times per second.

Connection pooling offers an elegant solution to this inefficiency by creating a reservoir of pre-established connections that your application can reuse, dramatically reducing overhead and improving performance. Instead of constantly opening and closing connections, your application simply borrows a connection from the pool when needed and returns it when finished, allowing that same connection to serve many subsequent requests.

Why Connection Pooling Matters

The performance benefits of connection pooling can be quite substantial. Here's why: establishing a new database connection typically takes between 50 to 100 milliseconds, which might not sound like much until you multiply it across thousands of requests. With connection pooling, your application can handle significantly more concurrent users because it's not wasting time and resources constantly creating and destroying connections. Additionally, connection pools protect your database server from being overwhelmed by too many simultaneous connections, which could cause it to slow down or even crash.

How to Configure Connection Pooling

The configuration of a connection pool requires careful attention to several key parameters:

• The minimum pool size determines how many connections remain open even when your application is idle, ensuring that some connections are always ready when traffic picks up.
• The maximum pool size sets an upper limit on how many connections can exist simultaneously, preventing your application from overwhelming the database server.
• Connection timeout settings specify how long the application should wait when requesting a connection from a pool that's currently at maximum capacity. If all connections are in use and none become available within this timeout period, the application will receive an error rather than waiting indefinitely.
• The idle timeout parameter determines how long a connection can sit unused in the pool before being closed, which helps free up resources during periods of low activity.

When configuring your pool, start conservatively. A good rule of thumb for the maximum pool size is to calculate the number based on your database server's capacity divided by the number of application instances that will connect to it. For example, if your database can handle 100 connections and you have five application servers, consider setting each application's maximum pool size to around 20 connections.

Common Mistakes to Avoid

One of the most frequent mistakes developers make is setting the pool size too large. While it might seem intuitive that more connections mean better performance, database servers actually work best with a moderate number of connections. Too many connections lead to excessive context switching and resource contention, ultimately degrading performance. Studies have shown that for many workloads, a pool size between 10 and 30 connections per application instance provides optimal throughput.

Another critical error is failing to properly return connections to the pool. If your application code opens a connection but doesn't close it due to an exception or programming oversight, that connection remains locked and unavailable to other parts of your application. Over time, this connection leak will exhaust your pool, causing new requests to timeout and fail. Always use try-finally blocks or equivalent constructs in your programming language to ensure connections are returned even when errors occur.

Developers sometimes also neglect to configure connection validation. Connections can become stale or broken due to network issues, database restarts, or timeout settings on the database server. Without validation checks, your application might retrieve a dead connection from the pool and fail when attempting to use it. Enabling connection testing ensures that the pool automatically detects and replaces broken connections before handing them to your application.

Monitoring Connection Pool Performance

Once you've configured connection pooling in your database, monitoring becomes essential to ensure your settings are appropriate for your workload. Tools like Navicat Monitor can help by tracking overall database connection activity from the server's perspective, showing you metrics like the current number of active connections, connection patterns over time, and when connection counts spike unexpectedly. While Navicat Monitor observes connections at the database server level rather than within your application's connection pool itself, this server-side view provides valuable insight into whether your pool sizing decisions are creating the right balance. If you notice that your database consistently shows connection counts near your server's maximum capacity, or if you see frequent connection spikes that correlate with application slowdowns, these patterns suggest your application connection pools may need adjustment. Combining this server-level monitoring with application-level metrics from your pooling library gives you a complete picture of how connections flow through your entire system, helping you identify bottlenecks and optimize performance effectively.

Conclusion

Database connection pooling represents one of those infrastructure decisions that often goes unnoticed when done correctly but can cause significant problems when misconfigured. By maintaining a ready supply of reusable connections, properly configuring pool parameters for your specific workload, and avoiding common pitfalls like oversized pools and connection leaks, you can dramatically improve your application's performance and reliability. The time invested in understanding and properly implementing connection pooling pays dividends in the form of faster response times, better resource utilization, and a more stable application overall.

Database credentials represent one of the most critical security assets in any organization. When these credentials fall into the wrong hands, the consequences can be devastating, from data breaches to regulatory fines and reputational damage. Understanding how to properly manage, store, and rotate these credentials is essential for maintaining a secure database environment.

Understanding Secrets Management

Secrets management refers to the tools, processes, and policies used to control access to sensitive authentication information. Rather than hardcoding passwords directly into application code or storing them in plain text configuration files, modern secrets management solutions provide encrypted storage with strict access controls. These systems act as centralized vaults where credentials can be stored, accessed programmatically, and audited comprehensively.

The fundamental principle behind effective secrets management is separation of concerns. Application code should never contain actual credentials but rather references to them. When an application needs to connect to a database, it requests the credentials from the secrets management system at runtime, uses them briefly for authentication, and then discards them from memory. This approach dramatically reduces the attack surface because credentials never persist in application code repositories or deployment packages.

Popular secrets management platforms like HashiCorp Vault, AWS Secrets Manager, and Azure Key Vault provide additional security layers through features like dynamic secrets generation, time-limited access tokens, and detailed audit logging. These systems ensure that every credential access is tracked, making it possible to identify suspicious patterns or unauthorized access attempts.

Implementing Credential Rotation Strategies

Credential rotation involves regularly changing passwords and access keys to limit the window of vulnerability if credentials become compromised. Without rotation, a single leaked password could provide indefinite access to your database. Establishing a rotation schedule based on your organization's risk profile is crucial, whether that means rotating credentials monthly, quarterly, or on-demand when security incidents occur.

Automated rotation is significantly more reliable than manual processes. Modern secrets management systems can automatically generate new passwords, update them in the database, and notify connected applications without requiring downtime. This automation eliminates the human error factor that often leads to security gaps during manual credential updates.

When implementing rotation, consider the impact on connected applications and services. Implementing a grace period where both old and new credentials remain valid temporarily can prevent service disruptions during the transition. Additionally, maintaining a clear inventory of all systems that use specific credentials helps ensure nothing gets overlooked during rotation cycles.

Avoiding Common Security Pitfalls

One of the most frequent mistakes organizations make is storing credentials in version control systems like Git. Even if a repository is private, this practice creates numerous copies of sensitive information across development machines and backup systems. Developers should use environment variables or configuration management tools instead, keeping credentials completely separate from source code.

Another critical pitfall involves insufficient access controls on credential storage locations. Configuration files containing database passwords should have restrictive file permissions, ensuring only the specific user account running the application can read them. Similarly, cloud storage buckets or secret management systems should enforce the principle of least privilege, granting access only to services and individuals who genuinely need it.

Default or weak passwords represent another common vulnerability. Many database installations ship with default administrative credentials that must be changed immediately upon deployment. Strong passwords should combine uppercase and lowercase letters, numbers, and special characters, with sufficient length to resist brute force attacks. Even better, consider using randomly generated passwords that humans never need to remember or type manually.

How Navicat Supports Secure Credential Management

Navicat, the popular database management and development tool, implements several security features to help protect your database credentials. When you save connection information, Navicat encrypts database passwords before storing them, ensuring that credentials aren't stored in plain text on your computer. The connection settings are saved in locations that only the logged-in user can access, preventing other users on the same system from viewing your database configurations.

For remote database connections, Navicat supports SSH tunneling, which establishes secure encrypted sessions between your client and the database server. This feature is particularly valuable when connecting to databases over untrusted networks, as it wraps all database traffic in an encrypted tunnel. You can authenticate these SSH connections using either passwords or public/private key pairs, with the latter providing stronger security against unauthorized access.

Navicat also includes support for SSL connections, allowing you to encrypt the communication channel between the client application and your database server. This prevents credentials and data from being intercepted during transmission. When working with Navicat Cloud, the service uses encryption both in transit through SSL connections and at rest through server-side encryption, though it's worth noting that database passwords themselves are never synchronized to the cloud, only connection settings.

Conclusion

Managing database credentials securely requires a comprehensive approach that combines proper secrets management infrastructure, regular credential rotation, and vigilance against common security mistakes. By treating credentials as critical assets that deserve dedicated protection mechanisms, organizations can significantly reduce their risk of data breaches and unauthorized access. The investment in proper credential management practices pays dividends through improved security posture, easier compliance with regulatory requirements, and greater peace of mind knowing that your data remains protected.

In today's fast-paced economy, database downtime can result in significant financial losses and damage to an organization's reputation. Building resilient database architectures has become indispensable for businesses that depend on continuous access to their data. A truly resilient database system can withstand failures, recover quickly from disasters, and maintain high availability even under adverse conditions.

Components of a Resilient Database

Database resilience refers to a system's ability to maintain operations during and after disruptions, whether they stem from hardware failures, software bugs, network issues, or natural disasters. A resilient architecture incorporates multiple layers of protection that work together to minimize downtime and data loss. This approach combines proactive planning with reactive capabilities, ensuring that when problems inevitably occur, their impact remains minimal and recovery happens swiftly.

Disaster Recovery Planning

Disaster recovery forms the foundation of database resilience by establishing procedures for restoring operations after catastrophic events. Effective disaster recovery begins with comprehensive backup strategies that capture both full and incremental snapshots of your data at regular intervals. These backups should be stored in geographically diverse locations to protect against regional disasters, with at least one copy maintained off-site or in a different cloud region.

Recovery Time Objective (RTO) and Recovery Point Objective (RPO) are critical metrics that guide disaster recovery planning. RTO defines the maximum acceptable downtime, while RPO determines how much data loss your organization can tolerate. Understanding these metrics helps you design appropriate backup frequencies and recovery procedures. Regular disaster recovery drills ensure that your team can execute recovery plans smoothly under pressure, revealing potential weaknesses before a real crisis occurs.

High Availability Strategies

High availability focuses on minimizing planned and unplanned downtime through redundancy and automated failover mechanisms. Database replication creates multiple copies of your data across different servers or data centers, allowing traffic to be redirected if the primary database becomes unavailable. Synchronous replication ensures data consistency across all replicas but may introduce latency, while asynchronous replication offers better performance at the cost of potential data lag.

Load balancing distributes database queries across multiple servers, preventing any single system from becoming overwhelmed. This not only improves performance but also provides redundancy, as other servers can absorb the workload if one fails. Implementing connection pooling and caching layers further enhances availability by reducing the load on your database servers and providing faster response times for frequently accessed data.

Chaos Engineering for Databases

Chaos engineering represents a proactive approach to resilience by deliberately introducing controlled failures into your database systems to identify weaknesses before they cause real problems. This practice involves running experiments that simulate various failure scenarios, such as server crashes, network partitions, or sudden traffic spikes, while monitoring how your system responds.

Starting with non-production environments, chaos experiments might include killing database processes, introducing network latency, or exhausting system resources to observe how replication handles these disruptions. Gradually expanding these experiments to production systems during low-traffic periods builds confidence in your architecture's resilience. The insights gained from chaos engineering lead to improvements in monitoring, alerting, and automated recovery procedures that strengthen your overall database infrastructure.

Navicat's Role in Database Resilience

Navicat provides comprehensive database management tools that support resilience through features like Data Synchronization, Data Transfer, and backups:

• The Data Synchronization feature helps maintain consistency across multiple databases, which is essential for high availability configurations. The tool allows you to synchronize data between databases and set up regular synchronization tasks, ensuring your replicas remain up to date.
• The Data Transfer feature facilitates smooth data migration between different database systems, minimizing risks of data loss or corruption during infrastructure changes or disaster recovery scenarios.
• Navicat's backup functionality creates structured snapshots of your databases that can be restored quickly when needed, supporting disaster recovery planning with its user-friendly interface for creating and managing database backups.

For monitoring and administration, Navicat Monitor provides real-time performance monitoring for your database server instances, helping you detect potential issues before they impact availability. The platform supports multiple database systems including MySQL, MariaDB, PostgreSQL and SQL Server. It's also compatible with cloud databases like Amazon RDS, Amazon Aurora, Oracle Cloud, Google Cloud and Microsoft Azure, making it valuable for organizations managing diverse database environments that need consistent resilience practices across different platforms.

Conclusion

Building resilient database architectures requires a comprehensive approach that combines disaster recovery planning, high availability strategies, and proactive testing through chaos engineering. By implementing multiple layers of protection and regularly testing your systems under stress, you create databases that can withstand failures and maintain operations even during adverse conditions. The investment in resilience pays dividends through reduced downtime, protected data, and the confidence that your critical systems can weather any storm.

Database licensing currently undergoing a significant transformation that will reshape how organizations budget for and deploy data infrastructure. Traditional perpetual licensing models, where organizations paid substantial upfront fees for indefinite database use, are giving way to subscription-based and consumption-driven approaches that promise greater flexibility but introduce new complexities. Simultaneously, the tension between open-core and fully open-source models is forcing organizations to reconsider their relationships with database vendors and the broader software strategy. Understanding these evolving licensing models has become essential for technology leaders making strategic decisions about their data infrastructure investments.

The Subscription Revolution

Subscription-based database licensing has emerged as the dominant model among major vendors, fundamentally changing the economics of database deployment. Rather than paying large capital expenditures for perpetual licenses, organizations now face predictable monthly or annual operational expenses. This shift aligns well with cloud adoption patterns and provides vendors with steady, recurring revenue streams. For organizations, subscription models lower initial barriers to entry and provide easier paths to scaling database capacity up or down based on changing needs. However, this approach introduces long-term cost considerations that differ markedly from traditional licensing, as organizations never truly own their database software and must maintain continuous payments to retain access. The total cost over a five or ten year period can significantly exceed traditional perpetual licensing costs, making careful financial modeling essential when evaluating subscription offerings.

Usage-Based Pricing and Serverless Models

Beyond simple subscriptions, usage-based pricing represents the next frontier in database licensing evolution. Serverless database offerings from cloud providers charge based on actual consumption of compute, storage, and I/O resources rather than pre-allocated capacity. This consumption model promises perfect alignment between costs and actual usage, eliminating waste from over-provisioned resources and allowing organizations to pay only for what they use. The appeal is particularly strong for workloads with variable or unpredictable demand patterns, where traditional capacity planning proves challenging. However, usage-based pricing introduces budget unpredictability that finance teams find difficult to manage, as database costs can fluctuate significantly month to month. Organizations adopting these models often find value in maintaining some fixed-cost components within their database infrastructure, such as management and development tools with stable licensing. Database management tools like Navicat, which offer perpetual licensing with optional maintenance, provide a cost anchor that helps balance the variability of consumption-based database pricing, making overall infrastructure costs more manageable and predictable for planning purposes.

Open Source Economics and the Open-Core Debate

The tension between fully open-source databases and open-core models has intensified as vendors seek sustainable business models while maintaining developer communities. Fully open-source databases like PostgreSQL and MySQL have gained enormous market share by eliminating licensing fees entirely, fundamentally disrupting the database market. Organizations adopting these technologies save dramatically on direct licensing costs while gaining freedom from vendor lock-in and the ability to modify source code for specific needs. However, the total cost of ownership extends beyond the database itself to encompass management, monitoring, and development tools. Commercial database management tools like Navicat provide enterprise-grade capabilities for open-source databases, filling gaps in native tooling and offering features such as visual schema design, automated comparison and synchronization, and cross-database migration tools. This combined approach, combining license-free databases with commercial management tools, often proves more cost-effective than traditional commercial database stacks while providing greater deployment flexibility.

The Open-Core Compromise

Open-core models attempt to balance community development with commercial viability by offering core database functionality under open-source licenses while reserving advanced enterprise features for paid editions. MongoDB, Elasticsearch, and Redis have all employed variations of this approach, with varying degrees of community acceptance. These models allow developers to build and test applications using free versions while requiring payment when organizations deploy at scale or need enterprise capabilities like advanced security, monitoring, or high availability features. The challenge lies in determining where to draw the line between free and paid features, as overly restrictive open-core models can alienate developer communities while overly generous ones struggle to generate sufficient revenue. Recent licensing changes by several open-core vendors, restricting cloud provider usage of their software, have highlighted the ongoing tension between openness and commercial sustainability in this space.

Vendor Independence and Strategic Flexibility

As licensing models evolve, organizations increasingly recognize the strategic value of maintaining vendor independence and deployment flexibility. Vendor lock-in concerns extend beyond licensing terms to encompass proprietary management tools, specialized skill requirements, and migration difficulties. Organizations seeking to preserve their options are adopting database-agnostic management platforms that provide consistent interfaces across multiple database technologies. Tools like Navicat enable database professionals to work seamlessly across MySQL, PostgreSQL, MongoDB, SQL Server, and Oracle using familiar workflows, significantly reducing the friction and retraining costs associated with changing database vendors or maintaining heterogeneous database environments. This approach gives organizations greater leverage in licensing negotiations, as the ability to migrate between platforms becomes more practical and less disruptive to operations.

Conclusion

The future of database licensing will likely see continued diversification rather than convergence on a single model. Organizations will navigate increasingly complex decisions that balance upfront costs against long-term expenses, flexibility against predictability, and vendor relationships against independence. Success in this shifting market requires holistic thinking that considers not just database licensing costs but the entire array of tools, skills, and architectural decisions that surround database deployments. Those who navigate these licensing decisions effectively will find opportunities to optimize costs while maintaining the flexibility to adapt as both technology and business requirements continue to evolve.

Supabase has rapidly emerged as one of the most popular open-source backend-as-a-service platforms in the developer community, earning its place among the top 100 most-starred repositories on GitHub. This impressive achievement reflects the platform's ability to simplify complex backend development while maintaining the power and flexibility that modern applications demand. This article explores what makes Supabase unique, its core capabilities, and how it integrates with professional database tools like Navicat to streamline your development workflow.

What is Supabase?

Supabase is a comprehensive Postgres development platform that provides developers with everything needed to build modern web, mobile, and AI applications. At its core, every Supabase project consists of a full PostgreSQL database, which brings over 35 years of proven reliability and feature robustness to your applications. The platform describes itself as an open-source alternative to Firebase, but with the added advantage of using SQL and the full power of Postgres rather than NoSQL databases.

The platform's philosophy centers on simplicity and developer experience. Rather than requiring developers to piece together multiple services and manage complex infrastructure, Supabase delivers an integrated solution where authentication, database APIs, real-time subscriptions, storage, and serverless functions work together seamlessly. This unified approach means developers can focus on building features rather than configuring backends.

Core Features and Capabilities

The strength of Supabase lies in its comprehensive feature set. The platform automatically generates RESTful APIs for your database through PostgREST, eliminating the need to manually create API endpoints for basic CRUD operations. This auto-generated API respects your database's Row Level Security policies, ensuring that data access remains secure and properly controlled.

Real-time functionality is built directly into the platform through Supabase Realtime, an Elixir server that monitors PostgreSQL's replication system and broadcasts changes over WebSockets. This makes building collaborative tools, live dashboards, and real-time chat applications straightforward, as data updates are automatically pushed to connected clients whenever database changes occur.

Authentication is handled by GoTrue, a JWT-based authentication system that supports multiple providers including email, phone, and social logins. The authentication system integrates tightly with Row Level Security, allowing developers to implement fine-grained access control policies directly in the database. Supabase Storage provides S3-compatible file storage with permissions managed through PostgreSQL, maintaining consistency across your entire application stack.

Edge Functions bring serverless capabilities to Supabase, allowing developers to write custom backend logic without managing servers. These functions run on Deno and can be deployed globally, with recent optimizations reducing function boot times by up to 300 percent in many cases. The platform has also introduced support for vector embeddings through the pgvector extension, positioning Supabase as a powerful option for AI applications that require semantic search and similarity matching.

Development Experience and Tooling

Supabase provides a sophisticated web-based dashboard that makes database management accessible even to developers who aren't PostgreSQL experts. The table editor presents a spreadsheet-like interface for viewing and editing data, while the SQL editor includes helpful features like query history and favorited queries. Recent updates have introduced tabbed interfaces in both the table and SQL editors, making it easier to work with multiple queries and tables simultaneously.

The platform's commitment to developer experience extends to its documentation and AI-powered assistance. Supabase recently launched an AI Assistant within the dashboard that can help with query optimization, schema design, and general troubleshooting. The platform also introduced postgres.new, a browser-based tool that uses large language models to help developers interact with PostgreSQL more intuitively.

How Navicat Supports Supabase

For developers who prefer working with dedicated database management tools, Navicat provides excellent support for Supabase databases. Navicat for PostgreSQL and Navicat Premium can connect directly to Supabase instances, offering a professional-grade interface for database administration and development that complements Supabase's built-in tools.

Connecting Navicat to Supabase is straightforward using the session pooler connection string available in your Supabase project settings. Navicat's intuitive graphical interface allows you to create, modify, and manage database objects like tables, views, functions, and triggers through visual designers rather than writing complex SQL. The latest version, Navicat 17.2, includes an AI Assistant, enhanced query visualization tools, and comprehensive data profiling capabilities that help you understand and optimize your Supabase database structure.

Navicat excels at tasks like data migration, allowing you to transfer data between Supabase and other database systems, and provides sophisticated backup and restore functionality. The visual query builder and execution plan analyzer are particularly useful for optimizing complex queries on your Supabase database. For teams working across multiple database platforms, Navicat Premium can manage Supabase alongside MySQL, MongoDB, SQL Server, and other databases from a single application, streamlining workflows for developers managing diverse data infrastructure.

Conclusion

Supabase represents a significant evolution in how developers approach backend infrastructure. By combining the reliability of PostgreSQL with modern developer tooling, real-time capabilities, and integrated authentication, the platform delivers a complete backend solution that scales from prototype to production. Its open-source nature ensures transparency and portability, while its growing ecosystem of tools and integrations, including support from established database management platforms like Navicat, demonstrates the platform's maturity and adoption. Whether you're building a startup MVP or an enterprise application, Supabase provides the foundation to move quickly without sacrificing the power and flexibility that complex applications demand.

Database performance and reliability have become critical determinants of business success, directly influencing revenue, customer satisfaction, and competitive positioning. Yet many organizations still rely on manual processes for critical database tasks like tuning, patching, and optimization. As databases grow in complexity and scale, the hidden costs of manual management compound rapidly. Database automation represents not just a technical upgrade, but a strategic investment with measurable returns that extend across the entire organization.

The Hidden Costs of Manual Database Management

Manual database administration carries costs that extend far beyond salary expenses. Database administrators spend countless hours on repetitive tasks such as applying security patches, monitoring performance metrics, and adjusting configuration parameters. This time investment translates directly to opportunity cost, as skilled DBAs are diverted from strategic initiatives that could drive innovation and business growth. Human error introduces another significant cost factor, with misconfigurations or delayed patches potentially leading to security breaches, data corruption, or system downtime. A single database outage can cost enterprises thousands or even millions of dollars per hour, not to mention the reputational damage that accompanies service disruptions.

Quantifying the Benefits of Automation

The business value of database automation becomes clear when examining specific operational improvements. Automated tuning continuously monitors database performance and adjusts parameters in real-time, eliminating the lag time between problem detection and resolution. Organizations implementing automated tuning typically see query performance improvements of twenty to forty percent, which translates directly to faster application response times and better user experiences. Automated patching reduces vulnerability windows from weeks to hours by deploying security updates immediately upon release, significantly decreasing the risk of exploitation. Furthermore, automation enables database teams to scale their operations without proportional increases in headcount, with some organizations managing three to five times more database instances per administrator after implementing automation tools.

Measuring ROI Across Key Metrics

The financial benefits of database automation appear in several interconnected areas of business operations. The most immediate savings come from reduced labor costs, as automation handles routine tasks that previously consumed thirty to fifty percent of DBA time. Downtime reduction represents another major financial benefit, with automated monitoring and self-healing capabilities preventing incidents before they impact users. Organizations should also consider the cost avoidance from improved security posture, as automated patching and compliance monitoring help prevent breaches that could result in regulatory fines and remediation expenses. Performance improvements drive revenue growth by enabling applications to handle higher transaction volumes and deliver superior customer experiences. When calculating total ROI, forward-thinking organizations include the strategic value of freeing technical talent to focus on innovation rather than maintenance.

Automating Database Tasks with Navicat

Navicat offers comprehensive automation capabilities through its Automation utility, which allows database administrators to create and schedule batch jobs for recurring tasks. The platform enables teams to automate queries, data imports and exports, data transfers between systems, and database synchronization operations.Organizations can set up automated data synchronization processes that run on predefined schedules, ensuring consistency across development, testing, and production environments without manual intervention.

Navicat's automation features include the ability to generate and export database documentation as PDFs on automated schedules, with email notifications sent to stakeholders upon completion. This streamlines compliance reporting and documentation requirements. The platform also supports automated backup operations through user-friendly interfaces for database-native backup utilities, helping teams maintain consistent data protection practices without relying on manual processes. By integrating automation into their workflows through tools like Navicat, database teams can achieve many of the ROI benefits discussed throughout this article while maintaining control and visibility over their database operations.

Conclusion

Database automation represents a compelling investment for organizations of any size. The quantifiable returns including reduced labor costs, minimized downtime, improved performance, and enhanced security create a strong financial case for automation. Beyond the immediate metrics, automation positions IT organizations for future growth by enabling teams to manage increasing database complexity without proportional resource increases. As databases continue to serve as the backbone of digital business, the question is no longer whether to automate, but how quickly organizations can implement automation to maintain competitive advantage.

Modern databases power everything from e-commerce platforms to healthcare systems, making their reliable performance absolutely critical to business operations. Yet traditional monitoring approaches that simply track CPU usage and memory consumption no longer meet the needs of today's complex data infrastructures. Database observability represents a fundamental shift in how organizations understand and optimize their database performance, transforming reactive troubleshooting into proactive performance management.

Monitoring vs. Observability

Traditional monitoring tells you that something is wrongperhaps response times have slowed or error rates have increased. Database observability, however, goes several steps further by helping you understand why problems occur and how to prevent them. This approach incorporates three essential pillars: metrics that quantify performance, logs that record system events, and traces that follow individual transactions through your infrastructure. Together, these elements provide the contextual insights needed to diagnose issues quickly and optimize performance continuously.

The difference becomes particularly apparent in distributed architectures where databases span multiple environments and interact with numerous applications. While traditional monitoring might alert you to slow response times, observability platforms can pinpoint the exact query causing bottlenecks, identify underutilized indexes, and even suggest optimization strategies based on historical patterns. This deeper visibility empowers database administrators to move from firefighting to strategic performance optimization.

The Evolution of Specialized Monitoring Tools

As database environments have grown more complex, specialized observability platforms have emerged to address these challenges. Leading solutions provide comprehensive visibility across multiple database engines, offering features such as query-level performance tracking, execution plan analysis, and automated anomaly detection. These platforms excel at correlating database performance with application metrics, helping teams understand how database issues impact overall user experience.

What makes modern tools particularly powerful is their ability to collect and analyze vast amounts of performance data in real time. They can track query execution patterns, monitor resource utilization across database clusters, and detect subtle performance degradations before they escalate into serious problems. Many platforms also incorporate machine learning algorithms that establish baseline performance profiles and automatically alert administrators when behavior deviates from normal patterns.

Navicat Monitor: Comprehensive Database Performance Management

Navicat Monitor exemplifies modern database observability through its agentless architecture that monitors MySQL, MariaDB, PostgreSQL, and SQL Server instances without requiring software installation on database servers. The platform supports both locally hosted instances and popular cloud services including Amazon RDS, Amazon Aurora, Oracle Cloud, Google Cloud, and Microsoft Azure, making it particularly valuable for organizations managing heterogeneous database environments.

The platform provides advanced root cause analysis capabilities that enable administrators to drill down into server metrics, performance statistics, hardware usage, and historical data when issues arise. Its built-in alert system allows administrators to define custom thresholds and receive notifications via email, SMS, SNMP, or Slack when warning or critical conditions occur, ensuring databases remain constantly available and performing optimally.

Navicat Monitor includes rich real-time and historical graphs that provide detailed views of server load, performance, availability, disk usage, network throughput, table locks, and replication health. The platform's Query Analyzer identifies long-running queries based on execution duration, wait types, CPU usage, and database read-write operations, allowing administrators to quickly identify and resolve performance bottlenecks. Users can also create custom metrics by writing their own queries to collect performance data specific to their needs and receive alerts when values exceed defined thresholds.

The Future of Database Performance Management

Database observability platforms represent a critical evolution in how organizations manage their data infrastructure. As databases continue to grow in complexity and importance, the deep visibility provided by observability tools becomes not just beneficial but essential. The integration of machine learning, automated diagnostics, and predictive analytics into these platforms promises even greater capabilities in the future, enabling truly proactive database management where potential issues are identified and resolved before they impact users. For organizations seeking to maintain competitive advantage in an increasingly data-driven world, adopting comprehensive observability solutions is no longer optional it's a strategic necessity!

A critical shortage of skilled database professional is threatening the digital transformation initiatives of organizations across a range of industries. As data volumes explode and database technologies proliferate, the demand for experienced database administrators, architects, and engineers has far outpaced the available talent pool. This skills gap has forced companies to rethink their approach to database management, accelerating the adoption of automation tools, low-code platforms, and productivity-enhancing technologies. Understanding this crisis and the strategies organizations are employing to address it has become essential for technology leaders across the globe.

The Root Causes of the Crisis

The database skills shortage stems from multiple converging factors. The rapid diversification of database technologies means organizations now operate heterogeneous environments running MySQL, PostgreSQL, MongoDB, Oracle, and SQL Server simultaneously, often alongside specialized systems for time-series data, graph databases, and vector search. Traditional approaches that relied on platform-specific specialists have become economically unfeasible, as finding experts for each database type proves nearly impossible in today's tight labor market. Compounding this challenge, experienced database professionals are retiring faster than new talent enters the field, while database complexity continues to increase with cloud migrations, distributed architectures, and real-time processing requirements.

Productivity Tools to the Rescue

In response to talent scarcity, organizations have increasingly turned to comprehensive database management tools that enable smaller teams to manage larger, more diverse database portfolios. Modern platforms provide visual interfaces that reduce dependency on command-line expertise while offering unified management across multiple database types. Tools like Navicat have become essential for organizations facing the skills gap, as they allow database professionals to work efficiently across MySQL, PostgreSQL, MongoDB, SQL Server, Oracle, and other platforms through a single consistent interface. Features such as visual query builders, automated schema comparison, and data modeling capabilities lower the barrier to entry for less experienced staff while maintaining the productivity of seasoned professionals.

Adapting Through Democratization

The shortage has accelerated the democratization of database management, with organizations enabling developers, data analysts, and other technical staff to perform tasks that previously required dedicated DBA expertise. This shift has been facilitated by intuitive database management platforms that abstract complexity without sacrificing capability. By providing graphical schema design tools, automated backup scheduling, and simplified data migration features, these platforms allow non-specialists to handle routine database operations competently. This democratization reduces bottlenecks created by DBA shortages and frees specialized database professionals to focus on complex architectural decisions, performance optimization, and strategic initiatives that truly require deep expertise.

Conclusion

The database skills gap represents a fundamental challenge for modern organizations, but it has also driven innovation in how database work is approached and distributed across technical teams. By combining strategic tool adoption with workforce development initiatives and organizational restructuring, companies are finding ways to maintain and even expand their database capabilities despite persistent talent shortages. Success in this environment requires embracing automation, investing in platforms that multiply individual productivity, and cultivating versatile database professionals who can work effectively across multiple technologies rather than specializing narrowly in single platforms.

Organizations today face increasingly complex decisions about where and how to deploy their database infrastructure. Multi-cloud database strategies, which involve distributing data systems across multiple cloud providers such as AWS, Azure, and Google Cloud, have emerged as a viable approach for enterprises seeking to balance cost, performance, and flexibility. Understanding the economic implications of these deployments is essential for making informed strategic decisions that align with both technical requirements and business objectives. With that in mind, today's blog article will cover important cost analysis considerations, how to avoid vendor lock-in, and more!

Cost Analysis and Optimization

The financial landscape of multi-cloud databases presents both opportunities and challenges. While a single-cloud approach might seem simpler from a cost management perspective, multi-cloud strategies can unlock significant savings through competitive pricing dynamics. Different cloud providers offer varying price points for similar database services, and organizations can leverage these differences by selecting the most cost-effective option for specific workloads. For instance, one provider might offer superior pricing for high-throughput transactional databases, while another excels in cost efficiency for analytics workloads.

However, the economic picture extends beyond simple service pricing. Data transfer costs between clouds, often called egress fees, can accumulate quickly and erode potential savings. Organizations must carefully model their data flow patterns and access requirements to avoid unexpected charges. Additionally, the operational overhead of managing multiple cloud environments requires investment in skilled personnel and sophisticated management tools, which must be factored into the total cost of ownership calculation.

Vendor Lock-In Avoidance

Perhaps the most compelling economic argument for multi-cloud databases lies in reducing dependency on any single cloud provider. Vendor lock-in creates significant business risk by limiting negotiating leverage and restricting architectural flexibility. When all database infrastructure resides with one provider, organizations may face unfavorable pricing changes with limited alternatives. A multi-cloud approach fundamentally changes this dynamic by maintaining genuine optionality.

The strategic value of avoiding lock-in extends beyond pricing negotiations. Technology landscapes evolve rapidly, and the leading database services or features today may not maintain their competitive advantage indefinitely. By maintaining infrastructure across multiple clouds, organizations can more easily adopt emerging technologies and services from different providers without undertaking massive migration projects. This architectural flexibility translates directly into economic value by enabling faster responses to market opportunities and reducing the risk of technical debt.

Strategic Considerations

Successful multi-cloud database deployment requires careful strategic planning that balances technical requirements with economic realities. Data residency and sovereignty regulations increasingly dictate where certain data must be stored, making multi-cloud approaches not just economically advantageous but sometimes legally necessary. Organizations operating globally may find that certain regions are better served by specific cloud providers due to data center proximity, regulatory compliance, or local partnership arrangements.

Performance considerations also play a crucial economic role. Distributing databases across multiple clouds can improve application resilience and reduce latency by placing data geographically closer to users. However, these benefits must be weighed against the complexity of maintaining data consistency and the potential costs of cross-cloud data synchronization. Organizations must develop clear decision frameworks that evaluate which workloads benefit most from multi-cloud deployment and which are better served by single-cloud simplicity.

Managing Multi-Cloud Databases with Navicat

Navicat Premium provides comprehensive support for managing databases across multiple cloud environments, offering simultaneous connections to MySQL, Redis, PostgreSQL, MongoDB, MariaDB, SQL Server, Oracle, Snowflake, and SQLite databases from a single application. The platform is compatible with major cloud databases including Amazon RDS, Amazon Aurora, Amazon Redshift, Microsoft Azure, Oracle Cloud, Google Cloud, MongoDB Atlas, and others, making it particularly valuable for organizations implementing multi-cloud strategies.

Navicat also offers advanced features including data modeling tools that support Relational, Dimensional, and Data Vault 2.0 methods, visual query builders, and the ability to synchronize connection settings, queries, and workspaces through Navicat Cloud. This centralized approach significantly reduces the operational complexity and associated costs of managing heterogeneous database environments across multiple cloud providers, allowing database administrators to work with consistent tools and interfaces regardless of the underlying cloud platform.

Conclusion

The economics of multi-cloud databases represent a highly nuanced calculation that extends well beyond simple price comparisons. While multi-cloud strategies offer genuine opportunities for cost optimization, vendor lock-in avoidance, and strategic flexibility, they also introduce complexities that require careful management. Organizations must approach multi-cloud database deployment with clear economic models that account for direct costs, operational overhead, and strategic value. When implemented thoughtfully with appropriate management tools and governance frameworks, multi-cloud database strategies can deliver substantial economic benefits while positioning organizations for long-term technical and business success. The key lies in treating multi-cloud not as a universal solution, but as a strategic option to be deployed where it creates genuine value.

For years, Raft and Paxos have been the foundational pillars of distributed consensus in database systems. These algorithms revolutionized how distributed databases could maintain consistency across multiple nodes, providing reliable ways to agree on data values even in the face of network partitions and node failures. However, as applications have become increasingly global and data volumes have exploded, the database community has recognized that traditional consensus algorithms, while robust, can create bottlenecks in performance and scalability.

The emergence of new consensus mechanisms represents a fundamental shift in how we think about distributed databases. Modern approaches are designed from the ground up to handle the unique challenges of globally distributed systems, where network latency between distant data centers can be measured in hundreds of milliseconds rather than single-digit values. These next-generation algorithms prioritize not just correctness, but also throughput, latency reduction, and efficient resource utilization across geographically dispersed infrastructure.

Multi-Leader and Leaderless Approaches

One of the most significant departures from traditional consensus algorithms is the move away from single-leader architectures. While Raft and Paxos rely on a leader node to coordinate writes, newer approaches embrace multi-leader or even leaderless architectures that can accept writes at multiple locations simultaneously. This architectural shift dramatically reduces write latency for globally distributed applications, as clients can write to the nearest data center without waiting for coordination with a distant leader node.

Conflict-free Replicated Data Types, or CRDTs, represent a particularly elegant solution to the consensus challenge. Rather than requiring nodes to agree on the order of operations before applying them, CRDTs are mathematical structures designed to converge to the same state regardless of the order in which operations are received. This allows databases to achieve eventual consistency without the coordination overhead of traditional consensus, enabling exceptional performance for use cases where temporary divergence is acceptable.

Optimistic Concurrency and Hybrid Models

Another frontier in distributed consensus involves optimistic concurrency control mechanisms that assume conflicts are rare and handle them as exceptions rather than as the norm. These systems allow transactions to proceed without extensive locking or coordination, validating consistency only at commit time. When combined with intelligent conflict resolution strategies, this approach can deliver remarkable performance improvements for workloads where contention is naturally low.

Hybrid consensus models are also gaining traction, intelligently selecting between different consistency levels and coordination mechanisms based on the specific requirements of each transaction. These adaptive systems might use strong consistency with traditional consensus for critical financial transactions, while employing looser consistency guarantees for less critical operations like user preference updates. This flexibility allows database systems to optimize for both correctness and performance across diverse workloads.

Working with Distributed Databases Using Navicat

As organizations adopt these sophisticated distributed database architectures, effective management tools become essential. Navicat, a comprehensive database management and development platform, provides excellent support for working with distributed database systems. Navicat Premium enables database administrators to simultaneously connect to multiple database platforms including MySQL, PostgreSQL, MongoDB, Redis, and cloud-based solutions, making it well-suited for managing distributed database deployments that often span multiple database technologies.

Navicat's compatibility with major cloud database services, including Amazon RDS, Amazon Aurora, Microsoft Azure SQL Database, Google Cloud SQL, and MongoDB Atlas, allows teams to manage distributed databases across different cloud providers from a single interface. The platform's data transfer and synchronization capabilities are particularly valuable for distributed systems, enabling administrators to migrate data and maintain consistency across geographically distributed nodes. With features like secure SSH tunneling and SSL connections, Navicat ensures that management operations remain secure even when working with databases distributed across multiple regions and cloud environments.

Conclusion

The landscape of distributed consensus algorithms continues to evolve rapidly, driven by the demands of global-scale applications and the architectural possibilities enabled by cloud infrastructure. While Raft and Paxos remain important foundations, the future belongs to more nuanced approaches that can adapt to varying requirements for consistency, performance, and availability. As these technologies mature, they promise to make truly global, highly responsive distributed databases accessible to a broader range of applications, fundamentally changing how we build data-intensive systems at planetary scale.

The journey of running databases in containerized environments has been a transformative one, marking a significant shift from the early days when Kubernetes was designed primarily for stateless applications. Today, containerized databases represent a mature technology stack that enables organizations to manage data workloads with the same agility and scalability they've come to expect from their application layers. This evolution has been driven by innovations in persistent storage, specialized orchestration tools, and a growing understanding of how to balance the dynamic nature of containers with the stability requirements of stateful data systems.

How StatefulSets Changed the Game

When Kubernetes first emerged in 2014, it excelled at managing stateless containerized applications but struggled with databases and other stateful workloads. The introduction of StatefulSets in Kubernetes 1.5 marked a pivotal moment in this evolution, providing the foundational features necessary for managing stateful applications. Unlike standard Deployments, StatefulSets maintain stable network identities for pods, ensure ordered deployment and scaling, and provide persistent storage that survives pod rescheduling. This means that each database instance receives a predictable hostname and storage volume that persists even when the pod moves between nodes, addressing one of the fundamental challenges of running databases in short-lived container environments.

StatefulSets also introduced ordered graceful deployment and scaling, which is critical for database clusters that require specific initialization sequences or leader election processes. When scaling a database cluster up or down, StatefulSets ensure that operations happen in a controlled, sequential manner rather than all at once, preventing data inconsistencies and ensuring that replication relationships remain intact throughout the process.

Operators: Bridging the Gap Between Kubernetes and Database Management

While StatefulSets provided the infrastructure foundation, Kubernetes Operators emerged as the intelligent layer that brings database-specific expertise into the orchestration process. Operators extend the Kubernetes API through Custom Resource Definitions, allowing administrators to define database-specific resources such as backup policies, replication configurations, and scaling strategies. These operators contain controller logic that continuously watches the state of database deployments and executes the necessary actions to maintain desired configurations through reconciliation loops.

The sophistication of modern database operators has transformed how teams approach database lifecycle management in Kubernetes environments. Rather than manually executing backup procedures or failover operations, operators automate these complex workflows with an understanding of database-specific requirements. For PostgreSQL deployments, operators can automatically handle streaming replication setup, while MongoDB operators understand sharding configurations and can orchestrate complex cluster topologies. This automation is particularly valuable because it encodes years of database administration expertise into code that runs continuously, catching issues before they become problems and ensuring that best practices are consistently applied.

The Persistent Storage Challenge

Perhaps no aspect of containerized databases has been more complex than persistent storage. Kubernetes initially relied on ephemeral storage that disappeared when pods terminated, which was fundamentally incompatible with database workloads where data durability is paramount. The evolution of Persistent Volumes and Persistent Volume Claims addressed this challenge by providing an abstraction layer between storage infrastructure and the applications consuming it. Storage Classes emerged to enable dynamic provisioning, allowing databases to request storage with specific performance characteristics without administrators needing to pre-provision volumes manually.

However, persistent storage in Kubernetes environments introduces challenges that extend beyond simple volume mounting. Performance considerations become critical when database workloads demand consistent IOPS and low latency that can vary significantly across different storage backends. Network-attached storage solutions must balance accessibility across nodes with the performance overhead of remote access, while local storage offers excellent performance but complicates pod scheduling and failover scenarios. Backup and disaster recovery strategies also require careful planning, as traditional approaches may not translate directly to containerized environments where volumes are dynamically provisioned and pods may be ephemeral.

Working with Containerized Databases Using Modern Tools

As containerized databases have matured, the choice of tools for managing and interacting with them has grown accordingly. Navicat, a comprehensive database management tool, can connect to and work with containerized databases running in Docker and Kubernetes environments. When databases are deployed in containers with properly exposed ports, Navicat connects to them just as it would to traditional database instances, using the container's mapped network ports or cluster service endpoints. The platform supports a wide range of database systems commonly deployed in containers, including MySQL, PostgreSQL, MongoDB, Redis, and many others, providing a familiar graphical interface for database administration tasks regardless of whether the underlying database runs in a container or on traditional infrastructure.

Additionally, Navicat itself offers containerized deployment options, with both Navicat Monitor and Navicat On-Prem Server available as Docker images that can be deployed in containerized environments. This flexibility allows organizations to maintain consistent tooling across both traditional and cloud-native architectures, managing containerized databases with the same robust feature set that Navicat provides for conventional deployments.

Conclusion

The maturation of containerized databases represents a remarkable achievement in cloud-native technology, transforming what was once considered impossible into a production-ready approach for managing data workloads. Through the introduction of StatefulSets, the development of sophisticated operators, and the evolution of persistent storage solutions, Kubernetes has evolved from a platform hostile to stateful workloads into one that can reliably run mission-critical database systems. While challenges remain around performance optimization, storage management, and operational complexity, the trajectory is clear: containerized databases are not just viable but increasingly preferred for organizations seeking the agility and consistency that cloud-native architectures provide. As tooling and best practices continues to mature, we can expect containerized databases to become the standard rather than the exception.

For decades, databases have been firmly planted on servers and in data centers, accessible only through network calls from client applications. WebAssembly (WASM) is fundamentally changing this equation by enabling database engines to run directly in browsers, edge computing environments, and serverless platforms with performance that rivals native applications. This technological convergence opens new possibilities for developers, from offline-first applications to distributed data processing at the network edge. In this article, we'll examine some concrete examples of WASM databases both new and traditional, and learn about the tools available for managing these distributed data workloads.

How WebAssembly Enables Database Portability

WebAssembly is a binary instruction format designed for efficient execution across different platforms. By compiling database engines to WASM, developers can achieve near-native performance while maintaining cross-platform compatibility. This means a single compiled database binary can run in a browser on Windows, macOS, Linux, or mobile devices without modification. The sandbox environment that WASM provides also enhances security, isolating database operations from the host system while still allowing rapid data processing. This combination of portability, performance, and security makes WASM an ideal target for database engines designed for modern, distributed computing scenarios.

Examples of WASM-First and WASM-Enabled Databases

Several databases have embraced WebAssembly to extend their reach. SQLite, one of the world's most widely used databases, has been compiled to WASM, enabling lightweight SQL execution in browsers and edge environments. DuckDB, a powerful analytical database optimized for OLAP workloads, offers WASM distributions for in-browser data analysis without server-side processing. These WASM-native options are purpose-built for edge and browser environments.

Beyond these specialized projects, traditional database engines have also developed WASM support. PostgreSQL can run in browser environments through WASM compilation, allowing developers to build sophisticated applications with full PostgreSQL compatibility. MySQL similarly has WASM implementations available, bringing familiar relational database capabilities to web applications. MongoDB, the popular NoSQL database, has explored WASM deployments for embedded scenarios. Redis, the in-memory data store, also supports WASM configurations, enabling fast caching and session management directly in edge environments.

Real-World Applications

The implications of WASM databases extend across multiple use cases. Web applications can now function offline with full data persistence, synchronizing when connectivity returns. Data analysts can perform complex queries on large datasets directly in the browser without uploading sensitive information to external servers. Edge computing platforms can process and filter data closer to users, reducing latency and bandwidth costs. Serverless functions gain the ability to perform sophisticated database operations within their resource constraints, previously requiring external database connections.

Managing WASM Databases with Navicat

As database deployments increasingly span traditional servers and WASM environments, developers need tools that work across this diverse landscape. Navicat, a widely-used database management platform, can work with several databases that have WASM implementations, including PostgreSQL, MySQL, MongoDB, and Redis. This capability allows developers to manage their databases through a familiar interface whether they're running in traditional data centers or in WASM environments, streamlining database administration and development workflows across modern application architectures.

Looking Forward

The convergence of databases and WebAssembly represents a significant shift in how data is processed and managed. As more database engines gain WASM support and developer tools mature, we can expect increasingly sophisticated applications that leverage distributed data processing, offline-first architecture, and edge computing. The future of databases is becoming less about location and more about capability, with WASM ensuring that powerful data processing is available wherever it's needed.

The intersection of artificial intelligence and cybersecurity has had a tremendous impact on how organizations protect their most valuable asset: data. As AI technologies become increasingly sophisticated, they present both unprecedented opportunities for enhanced database security and novel threats that traditional protection mechanisms struggle to address. Database administrators now face the dual challenge of defending against AI-powered attacks while leveraging AI itself to strengthen their security posture.

Advanced Threat Detection Through AI

Modern databases are incorporating machine learning algorithms that continuously analyze access patterns, query behaviors, and data flows to identify anomalies that might indicate a security breach. These AI-driven systems can detect subtle deviations from normal operations that would be nearly impossible for human administrators to spot. By establishing baseline behaviors for users, applications, and network traffic, machine learning models can flag unusual activities in real-time, such as unauthorized access attempts, abnormal data exfiltration patterns, or suspicious query structures that might indicate SQL injection attempts.

The advantage of AI-powered threat detection lies in its ability to learn and adapt. Unlike static rule-based systems, these intelligent solutions continuously refine their understanding of what constitutes normal versus suspicious behavior. They can identify zero-day threats and novel attack vectors by recognizing patterns that deviate from established norms, even when those patterns don't match any known attack signatures.

AI-Powered Attack Prevention

Beyond detection, artificial intelligence enables proactive defense mechanisms that can prevent attacks before they compromise data integrity. Predictive analytics models assess risk factors across the database environment, identifying vulnerabilities and prioritizing remediation efforts based on potential impact. AI systems can automatically implement security policies, adjust access controls dynamically based on risk assessments, and even simulate attack scenarios to test defense mechanisms.

These prevention systems also combat the growing threat of AI-generated attacks, where malicious actors use machine learning to craft more sophisticated phishing campaigns, develop polymorphic malware, or automate the discovery of system vulnerabilities. By employing AI to understand and predict adversarial AI tactics, organizations can stay one step ahead of attackers who are themselves leveraging automation and intelligence.

Securing Database Connections with Navicat

Database management tools play a critical role in maintaining security throughout the development and administration lifecycle. Navicat provides several robust features designed to protect database connections and prevent unauthorized access. The platform supports SSH Tunneling and SSL/TLS encryption, which guarantee the confidentiality, integrity, and availability of data as it travels between the client and database server. This encryption ensures that even if network traffic is intercepted, the data remains unreadable to unauthorized parties.

Navicat also implements advanced authentication methods that provide multiple layers of protection against unauthorized access. These include PAM (Pluggable Authentication Modules), LDAP (Lightweight Directory Access Protocol), Kerberos authentication, Multi-Factor Authentication (MFA), and Single Sign-On (SSO) capabilities. This diversity of authentication options allows organizations to implement security policies that align with their specific compliance requirements and risk profiles, ensuring that only verified users can access sensitive database resources.

Conclusion

As artificial intelligence continues to evolve, so too will the cybersecurity landscape. Organizations must embrace AI-powered security solutions while remaining vigilant against AI-generated threats. The databases of tomorrow will need to be intelligent, adaptive, and resilient, capable of defending themselves against increasingly sophisticated attacks while enabling legitimate users to work efficiently and securely. Success in this new era requires not just technological investment, but a comprehensive strategy that combines advanced tools, robust policies, and continuous education about emerging threats and defense mechanisms.

The metaverse is a place where virtual reality, augmented reality, and persistent digital worlds converge and millions of users interact simultaneously. As these immersive environments evolve from concept to reality, they are exposing fundamental limitations in traditional database architectures and driving innovation in how we store, query, and synchronize data at unprecedented scales.

The Spatial Database Revolution

Traditional databases were designed for tabular data organized in rows and columns, but the metaverse operates in three-dimensional space. Every virtual object, avatar, and environmental element exists at specific coordinates in a 3D world, creating an immediate need for spatial databases that can efficiently handle geometric data. These specialized systems must answer queries like "find all users within 50 meters of this location" or "identify objects intersecting this boundary" in milliseconds, not seconds. The challenge extends beyond simple coordinate storage to encompass complex spatial relationships, collision detection, and proximity-based interactions that form the foundation of believable virtual experiences.

Real-Time Synchronization at Scale

Perhaps no challenge is more critical to metaverse databases than real-time synchronization. When thousands of users occupy the same virtual space, every movement, interaction, and state change must propagate to all relevant clients with minimal latency. Traditional database replication strategies, which might synchronize data every few seconds or minutes, simply cannot support the fluid experiences users expect. Instead, metaverse platforms require event-driven architectures with pub-sub messaging patterns, conflict resolution algorithms, and sophisticated caching layers that maintain consistency without sacrificing performance. The technical complexity multiplies when considering global deployments where users may be separated by continents yet share the same virtual room.

Supporting Massive Numbers of Concurrent Users

Conventional databases struggle when hundreds of users access the same data simultaneously. The metaverse amplifies this challenge exponentially, potentially requiring support for tens of thousands of concurrent users in a single instance. This demand has accelerated adoption of distributed database architectures that partition data across multiple nodes, employ sharding strategies based on spatial regions, and implement read replicas to distribute query loads. However, distribution introduces its own complications around data locality, cross-shard queries, and maintaining transactional integrity across a distributed system. Database architects must balance horizontal scalability against the need for strong consistency guarantees in financially critical transactions like virtual asset purchases.

Leveraging Navicat for Spatial Database Management

As organizations build out their metaverse infrastructure, tools like Navicat provide essential support for managing the complex database requirements these platforms demand. Navicat Premium offers unified access to PostgreSQL databases, which serve as a foundation for many spatial implementations through the PostGIS extension. Navicat supports connection to cloud-hosted databases including Amazon RDS, Azure Database for PostgreSQL, and Google Cloud SQL, enabling developers to manage distributed metaverse databases from a single interface. Its visual query builder and data modeling capabilities help teams design efficient schemas for spatial data, while its support for Redis provides critical tools for managing the in-memory caching layers that underpin real-time synchronization. With Navicat Data Modeler, database architects can visualize and optimize their spatial database structures before deployment, ensuring efficient indexing strategies for geospatial queries.

Conclusion

The metaverse is fundamentally reshaping database requirements, pushing the industry toward solutions that prioritize spatial awareness, real-time performance, and massive concurrency. As these virtual worlds mature from experimental platforms to mainstream destinations, the database technologies supporting them must also continue to evolve, incorporating lessons from gaming, distributed systems, and geospatial computing to create the high-performance foundations these immersive experiences demand.

The explosive growth of location-aware applications has ushered in a new era of geospatial database capabilities. What once required specialized Geographic Information Systems (GIS) and complex data processing pipelines can now be accomplished directly within mainstream database platforms like MySQL, SQL Server, and PostgreSQL. This renaissance represents a seismic shift in how organizations store, query, and analyze location-based data, opening doors to more sophisticated mapping, logistics optimization, and Internet of Things (IoT) applications.

The Evolution of Mainstream Database Geospatial Support

Traditional relational databases were designed primarily for structured, non-spatial data. However, the widespread adoption of location-based services and mobile applications has driven database vendors to integrate native spatial capabilities. Major platforms like PostgreSQL with PostGIS, MySQL's spatial extensions, Microsoft SQL Server's spatial data types, and Oracle Spatial have transformed ordinary databases into powerful geospatial engines.

This integration brings several advantages that weren't possible with separate GIS systems. Organizations can now perform complex spatial queries alongside traditional business data operations within a single database transaction. For example, a retail company can simultaneously analyze customer demographics, inventory levels, and store proximity in one unified query, eliminating the need for data synchronization between unrelated systems.

The performance improvements are equally significant. Modern spatial indexing techniques, such as R-trees and grid-based indexes, enable rapid querying of millions of geographic features. These advances make real-time location services feasible at unprecedented scales, supporting everything from ride-sharing applications to supply chain optimization systems.

Specialized Geospatial Database Solutions

While mainstream databases have gained spatial capabilities, specialized geospatial databases continue to push the boundaries of what's possible with location data. These purpose-built systems excel in scenarios requiring extreme performance, advanced spatial analytics, or handling of complex geographic data types that general-purpose databases struggle with.

Graph databases with spatial extensions, such as Neo4j's spatial procedures, excel at routing and network analysis problems. They can efficiently model transportation networks, utility infrastructures, and social relationships with geographic components. Similarly, time-series databases with spatial capabilities handle streaming location data from IoT devices, enabling real-time tracking and analysis of moving objects.

Distributed spatial databases address the challenges of managing massive geospatial datasets across multiple nodes. These systems can partition data geographically, ensuring that queries affecting specific regions are processed efficiently without unnecessary network overhead. This capability proves crucial for global applications serving users across different continents.

Applications Driving the Renaissance

The mapping and navigation industry represents the most visible application of modern geospatial databases. Companies like Google, Apple, and HERE process billions of location queries daily, requiring databases that can handle complex routing calculations, real-time traffic analysis, and point-of-interest searches with sub-second response times. These applications demand not just storage efficiency but also sophisticated query optimization for multi-dimensional spatial data.

Logistics and supply chain management have become increasingly sophisticated through geospatial database integration. Modern warehouse management systems use spatial databases to optimize picking routes, while delivery companies leverage geographic algorithms for dynamic route planning that adapts to real-time traffic conditions and delivery priorities. The integration of spatial and temporal data enables four-dimensional optimization that considers location, time, vehicle capacity, and delivery windows simultaneously.

IoT applications represent perhaps the fastest-growing segment driving geospatial database innovation. Smart city initiatives collect massive streams of location-tagged sensor data, from traffic monitors to environmental sensors. These applications require databases capable of ingesting high-velocity spatial data while simultaneously serving complex analytical queries for urban planning and real-time decision making.

Navicat's Geospatial Data Management Features

Navicat Premium, which supports connections to multiple database systems - including PostgreSQL, MySQL, SQL Server, and Oracle - provides comprehensive tools for managing geospatial data across different platforms. The software's unified interface allows developers and analysts to work with spatial data regardless of the underlying database technology, eliminating the learning curve associated with platform-specific tools.

The visual query builder simplifies the creation of complex database queries without requiring deep knowledge of SQL syntax. Users can construct queries through an intuitive graphical interface, which proves particularly valuable for teams where not all members have extensive database programming experience. When working with databases that contain geospatial data, this visual approach can help users navigate the additional complexity that spatial queries often involve.

Navicat's data modeling capabilities allow users to design database schemas through visual ER diagrams that represent table relationships and database structure. The tool provides reverse engineering functionality to load existing database structures and create visual models, along with the ability to generate documentation for database designs. These features prove valuable when working with any complex database schema, including those that incorporate geospatial data alongside traditional business data.

Conclusion

The geospatial database renaissance represents more than just technological advancement; it signifies a fundamental shift toward location-aware computing as a standard capability rather than a specialized niche. As IoT devices proliferate and mobile applications become increasingly sophisticated, the ability to efficiently store, query, and analyze spatial data within mainstream database systems will become even more critical.

Organizations that embrace these enhanced capabilities today position themselves to leverage location intelligence as a competitive advantage. Whether optimizing delivery routes, analyzing customer behavior patterns, or managing smart city infrastructure, the convergence of spatial and traditional data analytics opens unprecedented opportunities for data-driven decision making. The tools and platforms supporting this renaissance continue to evolve, promising even more powerful and accessible geospatial capabilities in the years ahead.

As the world's economies become increasingly data-driven, organizations have begun to recognize that their competitive advantage lies not just in collecting data, but in their ability to access, share, and monetize diverse datasets securely. Database marketplaces have emerged to facilitate this exchange, enabling organizations to unlock new revenue streams while maintaining stringent security standards.

The Rise of Database Marketplaces

Database marketplaces are sophisticated platforms that serve as intermediaries between data providers and data consumers, creating environments where organizations can securely buy, sell, and share datasets. These platforms function much like traditional marketplaces, but instead of physical goods, they facilitate the exchange of valuable data assets. The global data marketplace platform market, which supports the buying and selling of various data types, was valued at USD 1.49 billion in 2024, with projections indicating significant growth to reach USD 5.73 billion by 2030.

Unlike simple file-sharing systems, database marketplaces provide comprehensive infrastructure that includes data cataloging, quality assessment, access controls, and monetization mechanisms. They enable organizations to discover relevant datasets, evaluate data quality before purchase, and integrate external data seamlessly into their existing analytics workflows. According to Gartner, by 2024, 90% of large organizations will use external data to enhance their analytics, marking a significant shift in how businesses approach decision-making.

Key Benefits and Value Proposition

The adoption of database marketplaces offers compelling advantages that extend far beyond simple data access. For data providers, these platforms create new revenue opportunities by monetizing previously underutilized data assets. Organizations can transform their internal datasets into valuable products, generating direct income while maintaining control over how their data is used and distributed.

Data consumers benefit from access to high-quality, curated datasets that would be impossible or prohibitively expensive to collect independently. Data vendors can share complete and custom datasets and databases for a fixed price via an S3 bucket, with buyers then having unlimited access and ownership of analysis-ready datasets. This broader access to data enables smaller organizations to compete with larger enterprises by leveraging external datasets to enhance their analytics capabilities.

The platforms also provide significant operational efficiencies by standardizing data sharing processes, reducing the time and complexity typically associated with data partnerships. Organizations no longer need to negotiate individual data-sharing agreements or build custom integration solutions for each data source.

Leading Platforms and Market Landscape

The database marketplace is made up of various specialized platforms, each serving different market segments and use cases. Snowflake Data Marketplace stands as one of the most prominent examples, leveraging Snowflake's cloud data platform to enable seamless data sharing without requiring data movement. This approach ensures that data remains secure within the provider's environment while still being accessible to authorized consumers.

Other significant players include AWS Data Exchange, which integrates with Amazon's broader cloud offerings, and specialized platforms like Datarade, which focuses on commercial data transactions. For thousands of companies including Google, BCG and Pepsico, Datarade Marketplace is the easiest way to find the right data sources. The market also includes emerging platforms like Opendatabay, which offers a vast collection of curated, synthetic, premium, and open datasets to fuel data analysis, AI, and LLM applications.

North America data marketplace platform market led the overall data marketplace platform industry in 2024, with a share of more than 35%, indicating the region's leadership in data monetization and sharing initiatives.

Security and Privacy Considerations

Security remains paramount in database marketplace operations, as organizations must balance data sharing benefits with the need to protect sensitive information. Modern platforms implement multiple layers of security, including advanced encryption, access controls, and audit trails that track all data interactions. These systems ensure that data providers maintain visibility into how their datasets are being used while protecting against unauthorized access.

Privacy considerations are equally critical, particularly with increasing regulatory requirements like GDPR and CCPA. Data marketplaces may provide individuals with options to control how their data is used and to opt out of having their data sold or shared, including providing clear privacy settings, preferences for data capture and storage, and opt-out mechanisms. Successful platforms implement privacy-by-design principles, ensuring that personal data protection is built into their architecture from the ground up.

Many platforms also support synthetic data generation and anonymization techniques, allowing organizations to share valuable insights without exposing sensitive personal information. This approach enables broader data sharing while maintaining compliance with privacy regulations.

How Navicat Facilitates Data Sharing

Navicat's comprehensive database management capabilities play a crucial role in enabling effective data sharing within database marketplaces. In March of 2025, support for Snowflake was also integrated, providing users with robust tools for managing cloud-based data solutions, making it particularly relevant for organizations participating in modern data marketplaces.

The platform's multi-database connectivity enables seamless data movement and synchronization across different database systems, which is essential when participating in data marketplaces that may use various underlying technologies. Navicat Premium enables seamless connections to multiple databases, including MySQL, PostgreSQL, MongoDB, MariaDB, SQL Server, Oracle, SQLite, Redis, and Snowflake, all from a single application.

Navicat's security features provide the foundation for safe data sharing practices. Advanced authentication methods, including PAM, LDAP, Kerberos, MFA, SSO, provide multiple layers of protection against unauthorized access, while SSH Tunneling and SSL/TLS guarantee the confidentiality, integrity, and availability of data. These security capabilities are essential when organizations need to ensure that their data sharing activities meet enterprise security standards.

Additionally, when logged into Navicat On-Prem Server, shared objects behave exactly like local ones and can be viewed, edited, and deleted directly in Navicat, facilitating collaborative data management that supports marketplace participation.

Looking Forward

Database marketplaces represent a fundamental shift in how organizations approach data as a strategic asset. As the market continues to mature, we can expect to see increased specialization, with platforms focusing on specific industries or data types. The integration of artificial intelligence and machine learning capabilities will also enhance data discovery and quality assessment processes, making it easier for organizations to find and evaluate relevant datasets.

The future success of database marketplaces will depend on their ability to balance accessibility with security, ensuring that data sharing becomes more efficient while maintaining the trust and compliance standards that organizations require. As these platforms evolve, they will likely become essential infrastructure for data-driven organizations, enabling new forms of collaboration and innovation that were previously impossible.

Since its inception about a quarter century ago, Infrastructure-as-Code (IaC) has revolutionized how we manage and deploy infrastructure resources. This approach treats infrastructure configuration as code by introducing version control, automated deployment, and consistent environments. Database-as-Code (DaC) extends these same principles to database schema management, bringing the benefits of version control and deployment automation to one of the most critical components of any application stack.

Some Database-as-Code Fundamentals

Database-as-Code represents a fundamentally new approach to database management that moves away from traditional manual practices. Instead of manually executing SQL scripts or using graphical tools to modify database schemas, DaC treats database structure and changes as code artifacts that can be versioned, reviewed, and deployed through automated pipelines.

We can compare it to building a house: with traditional database management, different contractors show up and make changes without blueprints or documentation. Meanwhile, Database-as-Code is like having detailed architectural plans that everyone follows, with every change documented and approved before implementation. This approach ensures that your database schema evolves predictably and consistently across all environments.

The core principle involves storing all database schema definitions, migration scripts, and configuration files in version control systems alongside your application code. This creates a single source of truth for your database structure and enables you to track exactly how your database has evolved over time.

Key Components and Implementation Approaches

Database-as-Code encompasses several essential components that work together to create a comprehensive database management strategy:

• Schema definitions form the foundation, typically written in SQL DDL (Data Definition Language) statements or domain-specific languages that describe table structures, indexes, constraints, and relationships.
• Migration scripts handle the transformation of your database from one version to another. These scripts are carefully crafted to be both forward and backward compatible when possible, ensuring smooth deployments and rollback capabilities. Each migration is numbered sequentially and contains both upgrade and downgrade instructions.
• Deployment automation ties everything together through continuous integration and continuous deployment (CI/CD) pipelines. These automated workflows validate schema changes, run tests against sample data, and deploy approved changes to target environments. The automation ensures that human error is minimized and that all environments remain synchronized.
• Version control integration allows teams to collaborate on database changes just like application code. Pull requests enable peer review of schema modifications, and branching strategies can be employed to manage feature development and hotfixes. This collaborative approach helps catch potential issues before they reach production environments.

Tools and Implementation Support

Modern database management tools have evolved to support Database-as-Code workflows effectively. Tools like Liquibase and Flyway provide frameworks for managing database migrations and schema versioning. These platforms offer database-agnostic approaches that work across multiple database systems while maintaining consistent workflows.

Cloud platforms and containerization technologies have also embraced Database-as-Code principles, offering managed services that integrate seamlessly with version control systems and deployment pipelines. These tools reduce the operational overhead of implementing Database-as-Code while providing enterprise-grade reliability and scalability.

Navicat Premium enhances Database-as-Code practices by providing tools that facilitate the management and generation of database schema and data using code-based approaches. Navicat supports DaC principles in several key ways:

SQL Generation from Visual Tools

Navicat's visual query builder, data modeling tools, and stored procedure builder allow users to design and manage database objects graphically. These visual operations are then translated into the corresponding SQL scripts, which can be version-controlled as part of a DaC workflow.

Code Snippets and Automation

The Code Snippets feature allows users to save and reuse common SQL statements and code blocks, promoting consistency and reducing manual coding. Additionally, features like batch jobs and automated data synchronization can be configured and scheduled, enabling automated database tasks that align with DaC principles.

Data Migration and Synchronization

Navicat offers streamlined wizards for data migration and synchronization, which can be utilized to manage data changes in a controlled and repeatable manner, a key aspect of DaC. The generated SQL scripts from these operations can also be incorporated into a version control system.

SQL Editor Features

The SQL editor in Navicat provides features like code completion, syntax highlighting, and SQL beautifier, enhancing the efficiency and quality of manually written SQL code. This supports the creation of clean and maintainable SQL scripts for DaC.

Data Modeling and Schema Export

Navicat Data Modeler allows for the visual design of database schemas and the export of these designs as SQL scripts. This provides a code-based representation of the database structure that can be versioned and deployed.

Conclusion

Database-as-Code represents a natural evolution of Infrastructure-as-Code principles, transforming database management from a manual, error-prone process into a reliable, automated system that supports modern development workflows. While the initial transition may seem challenging for organizations with established processes, the long-term benefits of improved consistency, traceability, and collaboration far outweigh the learning curve. By adopting Database-as-Code practices, organizations can transform their database management from a manual, error-prone process into a reliable, automated system that supports modern development workflows and business requirements.

Modern organizations often find themselves managing information across multiple database systems, each serving different purposes and storing various types of data. Traditional approaches require separate connections and queries for each database, creating complexity and inefficiency. Cross-database query engines have emerged as powerful solutions to these issues, enabling seamless data integration and analysis across diverse storage systems through a single SQL interface.

How Cross-Database Query Engines Work

Cross-database query engines are specialized software platforms that provide a unified SQL interface for querying data across multiple, heterogeneous data sources simultaneously. Think of these engines as universal translators that can speak to different database languages while presenting a consistent interface to users. They abstract away the complexity of individual database systems, allowing data analysts and engineers to write standard SQL queries that can retrieve and combine data from various sources including relational databases, NoSQL systems, cloud storage, and even streaming data platforms.

The fundamental architecture of these engines typically involves a coordinator node that receives SQL queries, parses them, and creates an execution plan. This plan is then distributed across worker nodes that connect to the actual data sources, retrieve the necessary data, and perform the required computations. The results are then aggregated and returned to the user, all while maintaining the illusion of querying a single, unified database.

Leading Cross-Database Query Engines

Trino, formerly known as Presto, stands as one of the most prominent cross-database query engines in the market today. Originally developed by Facebook to handle their massive data analytics needs, Trino excels at interactive analytics and can query data sources ranging from traditional MySQL and PostgreSQL databases to modern systems like Apache Kafka, Amazon S3, and Elasticsearch. Its distributed architecture allows it to process queries across petabytes of data with impressive performance characteristics.

Apache Drill represents another significant player in this space, designed with a schema-free approach that allows users to query data without requiring predefined schemas. This flexibility makes Drill particularly valuable when working with semi-structured data formats like JSON, Parquet, and Avro files. Drill's self-service data exploration capabilities enable users to start analyzing data immediately without waiting for database administrators to define table structures.

Other notable engines include Apache Spark SQL, which combines cross-database querying with powerful data processing capabilities, and Dremio, which focuses on self-service data analytics with an emphasis on data virtualization and acceleration.

Key Benefits and Use Cases

Cross-database query engines deliver several compelling advantages that address common data management challenges. First, they dramatically simplify data integration by eliminating the need to move data between systems before analysis. This approach, known as data virtualization, reduces storage costs and ensures that users always work with the most current data available.

Performance benefits emerge from the engines' ability to push computations down to the data sources themselves, minimizing data movement across networks. Advanced query optimization techniques, including predicate pushdown and intelligent join ordering, ensure that queries execute efficiently even when spanning multiple systems.

From a business perspective, these engines accelerate time-to-insight by removing technical barriers that previously required extensive ETL (Extract, Transform, Load) processes. Data analysts can focus on deriving insights rather than wrestling with data integration challenges. Common use cases include real-time dashboards that combine transactional and analytical data, compliance reporting that aggregates data from multiple business systems, and exploratory data analysis that requires access to diverse data sources.

Navicat Premium for Cross-Database Management

Navicat Premium serves as an excellent complementary tool for organizations implementing cross-database query strategies. While cross-database query engines handle the heavy lifting of distributed query execution, Navicat Premium provides a user-friendly graphical tool for managing multiple database connections and performing cross-database operations. The platform supports a wide variety of different database types, allowing users to establish connections to various systems from a single interface.

Navicat Premium's cross-database query capabilities enable users to write and execute queries that span multiple databases without requiring the complex setup of dedicated query engines. For smaller-scale operations or development environments, this functionality provides immediate value. Additionally, Navicat's data synchronization and migration tools complement query engines by facilitating the movement and harmonization of data structures across different systems when needed.

Conclusion

Cross-database query engines represent a transformative approach to modern data analytics, breaking down traditional barriers between disparate systems and enabling organizations to derive insights from their complete data landscape. As data continues to grow in volume and variety, these engines will become increasingly essential for maintaining competitive advantage through data-driven decision making. The combination of powerful distributed query engines with intuitive management tools like Navicat creates a winning combination that empowers users to unlock the full potential of their organizational data assets.

Database performance has always been the backbone of successful applications, but traditionally, keeping databases running at peak efficiency has required the expertise of seasoned database administrators working around the clock. Now, artificial intelligence is able to automate database tuning systems by optimizing your database configurations, index strategies, and query execution plans without human intervention. This article explores how these intelligent systems work, examines their practical benefits for modern organizations, and discusses why combining automated optimization with human expertise creates the most effective approach to database performance management.

The Challenges of Traditional Database Tuning

Before we dive into the automated solutions, let's establish why database tuning has historically been such a complex undertaking. A database is not unlike a busy restaurant kitchen during the dinner rush. The kitchen staff needs to coordinate perfectly knowing which ingredients to prep, how to arrange workstations, and which orders to prioritize to serve customers efficiently. Similarly, databases must juggle multiple concurrent queries, manage memory allocation, and decide how to access data most efficiently.

Traditional database tuning requires administrators to manually analyze performance metrics, identify bottlenecks, and adjust dozens of configuration parameters. This process demands deep expertise and constant vigilance, as database workloads can shift dramatically throughout the day. A configuration that works perfectly during morning batch processing might cause significant slowdowns when interactive users flood the system in the afternoon.

How AI-Powered Database Tuning Works

Automated database tuning systems function like having an incredibly observant and quick-learning assistant who never sleeps. These AI-powered solutions continuously monitor your database's performance characteristics, analyzing patterns in query execution, resource utilization, and response times. The system builds a comprehensive understanding of your database's behavior under different conditions, much like how a seasoned driver learns to navigate traffic patterns on their daily commute.

The artificial intelligence component employs machine learning algorithms to identify optimization opportunities that might escape human notice. For instance, the system might discover that creating a composite index on seemingly unrelated columns dramatically improves performance for a specific subset of queries that run frequently during certain hours. These insights emerge from analyzing vast amounts of performance data that would be overwhelming for human administrators to process manually.

When the system identifies an optimization opportunity, it can automatically implement changes such as adjusting buffer pool sizes, modifying query execution strategies, or creating new indexes. Crucially, these systems include safety mechanisms that allow them to roll back changes if performance degrades, ensuring that automated improvements never compromise system stability.

The Benefits of Continuous Optimization

The advantages of automated database tuning extend far beyond simply reducing administrative overhead. Consider how your smartphone automatically adjusts screen brightness based on ambient lighting automated database tuning provides similar adaptive intelligence for your data infrastructure. The system responds to changing workload patterns in real-time, optimizing performance for current conditions rather than relying on static configurations that might have been appropriate weeks or months ago.

This continuous optimization approach proves particularly valuable for organizations with fluctuating workloads. An e-commerce platform, for example, might experience dramatically different database usage patterns during holiday shopping seasons compared to typical business periods. Automated tuning systems adapt seamlessly to these variations, ensuring optimal performance regardless of load characteristics.

Additionally, automated systems can identify and resolve performance issues before they impact end users. By analyzing trends and patterns, these solutions often detect emerging bottlenecks and implement preventive measures, much like how modern cars can predict when maintenance will be needed based on driving patterns and component wear.

Why the Human Touch Still Matters

Despite the impressive capabilities of automated database tuning systems, they don't completely replace the need for manual oversight and optimization. While automated database tuning systems handle routine operations excellently, experienced administrators remain essential for complex situations and strategic decision-making by bringing contextual understanding that automated systems cannot fully replicate. They understand business requirements, anticipate upcoming changes in application usage patterns, and can make strategic decisions about database architecture that go beyond performance optimization. For instance, a DBA might recognize that certain performance issues stem from fundamental design problems that require application-level changes rather than database tuning.

This is where specialized monitoring tools like Navicat Monitor prove invaluable for bridging the gap between automated optimization and human expertise. Navicat Monitor provides database professionals with comprehensive performance monitoring and analysis capabilities that complement automated tuning systems. The platform enables administrators to build custom metrics that track specific performance indicators relevant to their database environments, while its Query Analyzer offers graphical representations of query logs and detailed performance statistics. When automated systems make recommendations or implement changes, Navicat Monitor's visualization tools and alert mechanisms help administrators understand the impact and rationale behind these optimizations, ensuring that human expertise remains an integral part of the database management process.

Conclusion

Automated database tuning represents a significant leap forward in how we manage database performance, offering the promise of continuously optimized systems that adapt to changing conditions without constant human intervention. While these AI-powered solutions handle routine optimization tasks with impressive efficiency, the combination of automated intelligence and human expertise creates the most robust approach to database management. As organizations increasingly rely on data-driven decision making, automated database tuning systems will become essential tools for maintaining the high-performance, reliable database infrastructure that modern applications demand.

In the world of web development, REST APIs have long been the standard for client-server communication. Now, a newer technology called GraphQL is reshaping how developers think about data retrieval and API design. Understanding GraphQL is becoming increasingly important for both web and developers developers who want to build efficient, flexible applications. For database developers specifically, GraphQL represents a fundamental shift in how applications interact with data stores. Rather than building multiple database queries to satisfy different API endpoints, GraphQL enables you to design your database schema in a way that directly mirrors your API structure. This alignment between database design and API consumption patterns means that database developers can create more intuitive, performant data access layers.

What is GraphQL?

GraphQL, which stands for "Graph Query Language", is both a query language for APIs and a runtime for executing those queries. Developed by Facebook in 2012 and open-sourced in 2015, GraphQL provides a more efficient, powerful, and flexible alternative to traditional REST API architectures.

You can compare GraphQL to a smart waiter at a restaurant. Instead of bringing you a pre-set meal (like REST endpoints that return fixed data structures), GraphQL lets you specify exactly what ingredients you want on your plate. You can request just the appetizer, or combine elements from different courses, all in a single request. This analogy helps illustrate GraphQL's core strength: giving clients precise control over the data they receive.

The "Graph" in GraphQL refers to how it models data as an interconnected network of relationships, much like how information connects in real-world scenarios. Rather than thinking in terms of multiple endpoints, GraphQL treats your entire API as a single, queryable graph of data.

Key Advantages of GraphQL

GraphQL addresses several pain points that developers commonly encounter with traditional REST APIs. The most significant advantage is the elimination of over-fetching and under-fetching data. With REST, you might request user information and receive everything about that user, even if you only need their name and email. GraphQL allows you to request exactly the fields you need, reducing bandwidth usage and improving performance.

Another major benefit is the reduction of multiple API calls. In REST architectures, displaying a user's profile with their posts and comments might require three separate requests. GraphQL enables you to fetch all this related data in a single query, significantly reducing network overhead and improving application speed.

GraphQL also provides strong typing and introspection capabilities. The schema acts as a contract between the client and server, clearly defining what data is available and how it can be queried. This self-documenting nature makes APIs easier to understand and work with, while the type system helps catch errors early in development.

How GraphQL Works

At its core, GraphQL operates through a schema that defines the structure of your data and the operations that can be performed. This schema serves as the single source of truth for your API, describing what data is available, how it's connected, and what operations clients can perform.

When a client makes a GraphQL query, it specifies exactly which fields it wants from which types. The GraphQL runtime then validates this query against the schema and executes it by calling resolver functions. These resolvers are responsible for fetching the actual data, whether from databases, other APIs, or any other data source.

The beauty of this approach lies in its flexibility. The same GraphQL endpoint can serve vastly different queries, each returning only the data requested. This eliminates the need for multiple endpoints while providing fine-grained control over data retrieval.

Working with GraphQL Using Navicat

While GraphQL offers powerful capabilities for API development, the effectiveness of any GraphQL implementation ultimately depends on the quality and performance of its underlying data sources. This is where database management tools like Navicat become essential.

Navicat excels at managing the diverse range of databases that commonly serve as GraphQL backends. Whether your GraphQL resolvers are fetching data from PostgreSQL, MySQL, MongoDB, or Redis, having robust database management capabilities is crucial for GraphQL success. You can use Navicat to optimize your database schemas, monitor query performance, and ensure your data structures are designed to efficiently support the queries that GraphQL applications often require.

The relationship between GraphQL and your database layer is particularly important to understand. Since GraphQL resolvers can trigger multiple database queries to fulfill a single API request, database performance becomes even more critical than in traditional REST architectures. Navicat's database monitoring and optimization features help you identify bottlenecks, optimize indexes, and structure your data in ways that minimize the database load when serving GraphQL queries.

Conclusion

GraphQL represents a significant evolution in API design, offering developers more control, efficiency, and flexibility than traditional approaches. By allowing precise data fetching, reducing network overhead, and providing strong typing, GraphQL addresses many of the challenges that have long plagued API development. As you explore this technology, tools like Navicat can significantly ease your development process, whether you're working directly with GraphQL APIs or managing the databases that support them. Understanding GraphQL is becoming essential for modern developers, and now is an excellent time to begin incorporating it into your development toolkit.

In todays AI and big data era, organizations are facing unprecedented complexity in how they manage, analyze, and secure their data. Database professionals are expected to deliver insights faster than ever while navigating fragmented environments across relational, NoSQL, and cloud-native systems. Fragmented tools often create inefficiency and errors, making it clear that what modern teams need is not just another tool, but a universal AI-driven platform that integrates and streamlines the entire database lifecycle. Solutions like Navicat Premium demonstrate how universal AI tools can help organizations thrive in this new era by boosting productivity, simplifying management, and empowering teams to make smarter decisions.

The Evolving Role of Database Professionals

Database administrators (DBAs) and developers juggle responsibilities like data modeling, query optimization, performance tuning, and security. These tasks are often repetitive, time-consuming, and prone to human errorespecially when working across multiple systems. AI-powered tools address these challenges by automating routine work, providing intelligent recommendations, and allowing professionals to focus on higher-value strategy and innovation.

The Case for a Universal AI Tool

Many organizations use multiple database platforms, BI tools, and admin dashboards. This fragmentation slows teams, creates inefficiencies, and complicates collaboration. A universal AI-driven platform consolidates these workflows into a single environment, enabling automation, natural language interaction, and predictive insights. By providing one hub for managing all database operations, universal AI tools reduce friction and accelerate results. Navicat Premium embodies this vision by unifying diverse databases while embedding AI-driven productivity features.

Streamlining Workflows with AI Automation

AI can automate schema design, query generation, and performance monitoring. Intelligent query builders analyze input and suggest optimized SQL queries, while predictive systems recommend index or schema adjustments before issues arise. This saves hours of manual debugging and ensures that teams are always working at peak efficiency.

Simplifying Multi-Database Management

With hybrid and cloud-native architectures, teams often manage relational and NoSQL databases simultaneously. A universal AI tool like Navicat Premium eliminates the need to switch between interfaces by offering a single platform to query, visualize, and synchronize data across systems. Natural language processing enables even non-technical staff to ask questions like, 'Show me the top customers this quarter,' and instantly see results.

Navicat Premium: Thriving in the AI Era

Navicat Premium supports MySQL, PostgreSQL, SQL Server, Oracle, Snowflake, MongoDB, Redis and cloud databases in one environment. Its AI-driven features simplify complex SQL creation through natural language, detect inefficiencies in database structures, and recommend optimizations like index adjustments. With automation for synchronization, backup, and cross-database migration, Navicat Premium positions itself as the universal AI-driven platform professionals can rely on.

Enhancing Decision-Making with Predictive Insights

Beyond automation, AI offers predictive analytics to anticipate performance bottlenecks or failures before they occur. Machine learning models trained on usage patterns help DBAs proactively tune systems, scale resources, and maintain reliability. This predictive capability ensures better cost management in cloud environments while minimizing downtime.

Improving Collaboration and Accessibility

AI-driven tools bridge the gap between technical experts and business users. Navicat Premiums visual dashboards and natural language interface enable analysts, managers, and DBAs to collaborate on insights without relying solely on SQL expertise. This democratizes access to data and accelerates decision-making across the organization.

Ensuring Data Security and Compliance

AI enhances security by flagging anomalies in real time, such as unusual logins or sensitive data queries. Navicats automated audit and compliance features reduce manual workload while helping teams stay aligned with regulations like GDPR and HIPAA.

Real-World Impact: A Case Study

A mid-sized e-commerce company managing multiple relational and NoSQL systems adopted Navicat Premium. The results were clear: AI-optimized queries reduced execution time by 30%, the unified interface eliminated tool-switching, and predictive alerts prevented downtime. Business users created their own reports through visual dashboards, freeing DBAs to focus on innovation. Together, these benefits saved the company hours each week while improving customer experience.

The Future of Universal AI Tools

As AI capabilities advance, the next frontier for universal tools like Navicat Premium lies in deeper BI visualization, more intuitive natural language interfaces, and smarter predictive analytics. In the future, universal AI-driven platforms wont just support teamsthey will guide them proactively, shaping strategies and keeping organizations ahead in the AI era.

Conclusion

In an age defined by AI and big data, fragmented tools are no longer enough. Organizations need universal AI-driven platforms that consolidate, automate, and empower. Navicat Premium (Free version is available) is at the forefront of this shift, enabling database professionals to work smarter, collaborate better, and deliver faster. The future belongs to teams equipped with universal AI toolsand Navicat is leading the way.

Conversational database interfaces represent a cutting edge approach to data interaction, powered by large language models that enable users to query databases using plain English rather than via complex SQL commands. Think of these interfaces as intelligent translators that sit between you and your database, converting your natural language questions into precise database queries and then presenting the results in an easily understandable format.

These systems leverage advanced natural language processing capabilities to understand context, intent, and nuance in human speech patterns. When you ask a question like "Show me all customers who made purchases over $1000 last month," the interface analyzes your request, identifies the relevant tables and columns, constructs the appropriate SQL query, executes it, and returns the results in a conversational manner. This technology levels the playing field by removing the technical barrier that has traditionally separated business users from their data. In this article, we'll explore how these revolutionary interfaces work, examines the key differences between conversational systems and NoSQL databases, and demonstrates how modern database management tools like Navicat support this technological innovation.

The Technology Behind Natural Language Queries

Large language models serve as the foundation for these conversational interfaces, having been trained on vast amounts of text data that includes both natural language and structured query languages. These models understand the relationships between everyday language and database operations, enabling them to perform complex translations between human intent and machine-executable commands.

The process involves several sophisticated steps that happen seamlessly in the background. First, the system parses your natural language input to identify key entities, relationships, and operations. Then it maps these elements to your specific database schema, understanding which tables contain the relevant information and how they relate to each other. Finally, it constructs and executes the appropriate query while handling potential ambiguities or errors gracefully.

Modern implementations often include context awareness, allowing for follow-up questions and maintaining conversation history. This means you can ask a follow-up question like "What about the previous year?" and the system understands you're referring to the same customer purchase data from your earlier query.

NoSQL versus Conversational Interfaces

Understanding the difference between NoSQL databases and conversational database interfaces is crucial for grasping how these technologies complement rather than compete with each other. This distinction often confuses newcomers to database technology because both represent departures from traditional database interactions, but they address entirely different aspects of data management.

NoSQL databases fundamentally change how data is stored and organized. Unlike traditional relational databases that store information in structured tables with predefined relationships, NoSQL systems embrace flexible, schema-less approaches. Document databases like MongoDB store information as JSON-like documents, while graph databases like Neo4j represent data as interconnected nodes and relationships. These systems excel at handling unstructured data, scaling horizontally across multiple servers, and adapting to changing data requirements without rigid schema constraints.

Conversational database interfaces, on the other hand, revolutionize how users interact with stored data, regardless of the underlying storage mechanism. These interfaces can work equally well with traditional SQL databases, NoSQL systems, or hybrid architectures. The key insight is that conversational interfaces address the user experience layer, while NoSQL addresses the data storage layer. You might have a conversational interface that allows natural language queries against a MongoDB document database, combining the flexibility of NoSQL storage with the accessibility of natural language interaction.

Leveraging Database Management Tools for Conversational Interfaces

Navicat provides comprehensive support for working with databases that implement conversational interfaces, offering a bridge between traditional database management and modern natural language query capabilities. The platform's intuitive design philosophy aligns perfectly with the accessibility goals of conversational database systems, providing visual tools that complement natural language interactions.

Through Navicat's unified interface, database administrators and developers can manage the underlying database structures that support conversational interfaces while also testing and refining the natural language processing capabilities. The tool's connection management features make it easy to work with various database systems that might be powering conversational interfaces, from traditional MySQL and PostgreSQL installations to modern NoSQL systems like MongoDB or cloud-based solutions.

Navicat's query building and visualization tools become particularly valuable when developing and debugging conversational database interfaces, allowing teams to understand exactly how natural language queries translate into database operations and optimize performance accordingly.

Conclusion

Conversational database interfaces powered by large language models represent a fundamental shift toward more accessible and intuitive data interaction. By removing the technical barriers traditionally associated with database queries, these systems enable broader organizational participation in data-driven decision making. As this technology continues to evolve, the combination of flexible storage solutions, intelligent query interfaces, and comprehensive management tools are making data truly accessible to users regardless of their technical expertise.

On Oct 28, 2024, the Seamless MySQL and MariaDB Management with Navicat On-Prem Server blog installment introduced an on-premise solution that allows distributed teams to collaborate in real time, share data, coordinate tasks, and communicate seamlessly through a centralized platform. It's one of two Navicat offerings for Collaboration - the other being Navicat Cloud. Whereas Navicat Cloud offers a central space for your team to store Navicat objects, Navicat On-Prem Server is an on-premise solution for hosting a cloud environment where you can securely store Navicat objects internally at your location.

Now, Navicat has just released Navicat On-Prem Server 3.0. It includes a few new features that promise to help foster even greater collaboration within your team. There are four main features as follows:

• Added support for PostgreSQL and Fujitsu Enterprise Postgres connections.
• The Query Editor has been enhanced to include Code Completion, Code Folding and SQL Beautify.
• Expanded Object Filtering Features.
• An improved "New Connection" Dialog.

In today's blog we'll be reviewing Navicat On-Prem Server 3.0 and evaluating how the above features help manage your MySQL, MariaDB, and PostgreSQL instances more effectively than ever before.

Upgrading to Version 3.0

Moving up to Navicat On-Prem Server 3.0 is a fairly simple process:

• To begin, head on over to the download page and locate the installation file that matches your operating system.
• Before running the installation program, be sure to uninstall the previous version of Navicat On-Prem Server.
• Now you're ready to execute the installation program.
• Once the installation completes, a browser with the initial setup page should open. If it doesn't, you can right-click the app icon in right bottom corner to start the app.
• Navicat On-Prem Server will remember your login credentials as well as all other configuration details, so you should be able to login to the app exactly as you did in previous versions.
• Upon logging in for the first time, you'll be greeted by the Welcome splash screen. It highlights all of the new features in version 3.0.

  

Now let's turn our attention to the new features described above.

PostgreSQL and Fujitsu Enterprise Postgres Connections

If we click the "+New" button at the top of the main app screen, a context menu will appear containing two items: "New Project" and "New Connection". Clicking on the latter will open the New Connection Dialog. Looking at the Connection Filter we can see that there are many new connection types, the most notable being PostgreSQL. Beyond that, the Vendor Filter has also been expanded to include Fujitsu, HighGo, Kingbase, and more:

Enhanced Query Editor

The Query Editor has been completely revamped to include some of the most sought-after features from the Navicat database administration and development tools. These include:

• Beautify SQL: Automatically reformat messy or compressed SQL code into clean, properly indented, and readable format with consistent spacing and line breaks with a click of a button! The resulting code will be structured in such a way that follows standard formatting conventions, making it easier to understand and maintain.
• Pin: Preserves critical data snapshots alongside their queries and timing information for ongoing analysis.

All of the essential features that you'd expect are also there, including syntax highlighting, code folding, code completion, and Query Explain.

The synergistic effect of all these features will be enhanced development productivity and clearer SQL code.

Expanded Object Filtering Features

Thanks to the expanded Object Filtering features, you can now efficiently search through large volumes of data to locate specific objects instantly, so important information is always within reach.

Improved "New Connection" Dialog

You'll notice that there are two new icons on the New Connection dialog to the left of the Search box: they activate Grid View and List View respectively. Previously, only Grid View was supported. Here's a comparison of each view type:

Conclusion

Navicat On-Prem Server 3.0 is the perfect solution for organizations who wish to benefit from the convenience and features of a cloud-based solution, all while maintaining full control over their data. Navicat On-Prem Server 3.0 allows teams to synchronize all their connection settings, queries, aggregation pipelines, snippets, model workspaces, BI workspaces and virtual group information across all their devices. Beyond sharing data, Navicat On-Prem Server 3.0 offers all the tools that professionals need for administering, monitoring, and managing MySQL, MariaDB, PostgreSQL, and Fujitsu Enterprise Postgres (FEP) databases.

Navicat On-Prem Server 3.0 is available for the Windows, macOS, and Linux operating systems. You'll find a 14-day trial version for each of these platforms on the download page.

On Oct 28, 2024, the Seamless MySQL and MariaDB Management with Navicat On-Prem Server blog installment introduced an on-premise solution that allows distributed teams to collaborate in real time, share data, coordinate tasks, and communicate seamlessly through a centralized platform. It's one of two Navicat offerings for Collaboration - the other being Navicat Cloud. Whereas Navicat Cloud offers a central space for your team to store Navicat objects, Navicat On-Prem Server is an on-premise solution for hosting a cloud environment where you can securely store Navicat objects internally at your location.

Now, Navicat has just released Navicat On-Prem Server 3.0. It includes a few new features that promise to help foster even greater collaboration within your team. There are four main features as follows:

• Added support for PostgreSQL and Fujitsu Enterprise Postgres connections.
• The Query Editor has been enhanced to include Code Completion, Code Folding and SQL Beautify.
• Expanded Object Filtering Features.
• An improved "New Connection" Dialog.

In today's blog we'll be reviewing Navicat On-Prem Server 3.0 and evaluating how the above features help manage your MySQL, MariaDB, and PostgreSQL instances more effectively than ever before.

Upgrading to Version 3.0

Moving up to Navicat On-Prem Server 3.0 is a fairly simple process:

• To begin, head on over to the download page and locate the installation file that matches your operating system.
• Before running the installation program, be sure to uninstall the previous version of Navicat On-Prem Server.
• Now you're ready to execute the installation program.
• Once the installation completes, a browser with the initial setup page should open. If it doesn't, you can right-click the app icon in right bottom corner to start the app.
• Navicat On-Prem Server will remember your login credentials as well as all other configuration details, so you should be able to login to the app exactly as you did in previous versions.
• Upon logging in for the first time, you'll be greeted by the Welcome splash screen. It highlights all of the new features in version 3.0.

  

Now let's turn our attention to the new features described above.

PostgreSQL and Fujitsu Enterprise Postgres Connections

If we click the "+New" button at the top of the main app screen, a context menu will appear containing two items: "New Project" and "New Connection". Clicking on the latter will open the New Connection Dialog. Looking at the Connection Filter we can see that there are many new connection types, the most notable being PostgreSQL. Beyond that, the Vendor Filter has also been expanded to include Fujitsu, HighGo, Kingbase, and more:

Enhanced Query Editor

The Query Editor has been completely revamped to include some of the most sought-after features from the Navicat database administration and development tools. These include:

• Beautify SQL: Automatically reformat messy or compressed SQL code into clean, properly indented, and readable format with consistent spacing and line breaks with a click of a button! The resulting code will be structured in such a way that follows standard formatting conventions, making it easier to understand and maintain.
• Pin: Preserves critical data snapshots alongside their queries and timing information for ongoing analysis.

All of the essential features that you'd expect are also there, including syntax highlighting, code folding, code completion, and Query Explain.

The synergistic effect of all these features will be enhanced development productivity and clearer SQL code.

Expanded Object Filtering Features

Thanks to the expanded Object Filtering features, you can now efficiently search through large volumes of data to locate specific objects instantly, so important information is always within reach.

Improved "New Connection" Dialog

You'll notice that there are two new icons on the New Connection dialog to the left of the Search box: they activate Grid View and List View respectively. Previously, only Grid View was supported. Here's a comparison of each view type:

Conclusion

Navicat On-Prem Server 3.0 is the perfect solution for organizations who wish to benefit from the convenience and features of a cloud-based solution, all while maintaining full control over their data. Navicat On-Prem Server 3.0 allows teams to synchronize all their connection settings, queries, aggregation pipelines, snippets, model workspaces, BI workspaces and virtual group information across all their devices. Beyond sharing data, Navicat On-Prem Server 3.0 offers all the tools that professionals need for administering, monitoring, and managing MySQL, MariaDB, PostgreSQL, and Fujitsu Enterprise Postgres (FEP) databases.

Navicat On-Prem Server 3.0 is available for the Windows, macOS, and Linux operating systems. You'll find a 14-day trial version for each of these platforms on the download page.

The database world is experiencing a memory-first revolution that's fundamentally changing how we approach data storage and processing. This transformation is happening from two directions simultaneously: traditional disk-based databases like PostgreSQL and MySQL are incorporating sophisticated in-memory capabilities, while pure in-memory systems like Redis are adding robust persistent storage features. The result is a new generation of hybrid databases that eliminate the age-old tradeoff between speed and reliability. This article explores how this revolution is reshaping the database landscape, from the driving forces behind the change to how to manage memory-first databases.

Why In-Memory Computing Matters

To appreciate this revolution, we need to understand why in-memory computing has become so crucial in modern data management. Traditional databases store data on disk, which requires time-consuming read and write operations every time you access information. Think of it like having to walk to a filing cabinet across the room every time you need a document, versus having all your important papers right on your desk.

In-memory computing keeps data in RAM, where it can be accessed thousands of times faster than disk storage. This dramatic speed improvement has made in-memory systems essential for applications requiring real-time analytics, high-frequency trading, gaming leaderboards, and session management. However, pure in-memory systems traditionally faced a critical limitation: data volatility. When power goes out or systems restart, everything stored only in memory disappears. Organizations have developed several strategies to mitigate this volatility risk while preserving the speed advantages of in-memory systems:

• Redundant in-memory clusters where data is replicated across multiple servers, ensuring that if one machine fails, the data remains available on other nodes.
• Periodic snapshots that capture the entire memory state to disk at regular intervals, much like taking photographs of your desk at the end of each day so you can restore it if everything gets scattered.
• Write-ahead logging, which records every data change to persistent storage before applying it to memory, creating a complete audit trail that can rebuild the memory state even after unexpected failures.

Adding Memory-First Capabilities to Traditional Database

Traditional databases like PostgreSQL, MySQL, and Oracle have recognized that modern applications demand faster response times than disk-based storage can provide. Rather than abandoning their proven architectures, these systems are integrating sophisticated in-memory layers that work seamlessly with their existing persistent storage.

Consider how PostgreSQL has evolved to include advanced caching mechanisms and in-memory table spaces. These features allow frequently accessed data to remain in memory while maintaining the database's ACID properties and durability guarantees. Similarly, MySQL's integration with memory engines and Oracle's in-memory column store demonstrate how traditional databases are adapting to meet performance demands without sacrificing their core strengths.

This evolution allows organizations to gradually adopt in-memory capabilities without completely overhauling their existing database infrastructure. They can identify performance-critical tables or queries and selectively apply in-memory optimizations while keeping the rest of their data in traditional storage. This hybrid approach provides a practical migration path that balances performance gains with operational stability.

Pure In-Memory Systems: Embracing Persistence

Meanwhile, pure in-memory systems like Redis, Memcached, and Apache Ignite are adding sophisticated persistence mechanisms. Redis, originally designed as a simple key-value store that lived entirely in memory, now offers multiple persistence options including point-in-time snapshots and append-only file logging.

These persistence features address the primary concern organizations have had with in-memory systems: data durability. Redis's RDB snapshots create periodic backups of the entire dataset, while AOF (Append Only File) logging records every write operation, allowing for complete data recovery even after system failures. These enhancements have transformed Redis from a simple caching solution into a full-featured database capable of serving as a primary data store for many applications.

The addition of persistence doesn't compromise the speed advantages of in-memory systems. Instead, it provides configurable durability options that let organizations choose the right balance between performance and data safety for their specific use cases. Applications can operate at memory speed while having confidence that their data will survive system restarts and failures.

In-Memory Database Management with Navicat

As databases evolve to support both in-memory and persistent storage capabilities, database administrators and developers need tools that can effectively manage these hybrid systems. Navicat provides comprehensive support for working with databases that embody this memory-first philosophy, offering a unified interface for managing both traditional and modern database architectures.

Navicat's support for Redis allows developers to work with in-memory data structures while configuring persistence settings, monitoring memory usage, and managing data expiration policies. The tool provides visual interfaces for understanding how data flows between memory and disk, making it easier to optimize performance while ensuring data durability. For traditional databases with in-memory capabilities, Navicat offers tools to monitor cache hit rates, configure memory allocation, and identify opportunities for in-memory optimization.

Conclusion

The memory-first database revolution represents a maturation of database technology that addresses the real-world needs of modern applications. Organizations no longer need to choose between speed and durability, or between familiar traditional databases and cutting-edge in-memory systems. This transformation is creating more flexible, efficient, and capable data management solutions that can adapt to diverse application requirements while reducing operational complexity. As this revolution continues, we can expect to see even more sophisticated hybrid systems that blur the lines between different database categories, ultimately providing better tools for managing the ever-growing demands of data-driven applications.

Database observability represents a totally new way for organizations to monitor and understand their data infrastructure. Unlike traditional monitoring that focuses on basic metrics like CPU usage and memory consumption, observability platforms provide deep, contextual insights into database behavior, enabling teams to understand not just what is happening, but why it's happening and how to optimize performance proactively. Today's blog explores the evolution from basic database monitoring to advanced observability, examining leading platforms, built-in database features, and practical implementation strategies for modern data environments.

Database Observability vs. Database Monitoring

Database observability extends beyond simple monitoring by incorporating three key pillars: metrics, logs, and traces. Think of it as the difference between checking your car's dashboard warning lights versus having a comprehensive diagnostic system that shows you engine performance, fuel efficiency patterns, and predictive maintenance needs. Observability platforms collect granular data about query execution plans, lock contention, index usage, and connection patterns, then correlate this information to provide actionable insights.

This approach becomes particularly valuable in modern distributed architectures where databases often span multiple environments and interact with numerous applications. Traditional monitoring might tell you that response times are slow, but observability platforms can pinpoint the specific query causing bottlenecks, identify which indexes are underutilized, and even suggest optimization strategies based on historical patterns.

Leading Database Observability Platforms

Several specialized platforms have emerged to address the growing complexity of database performance management. Datadog's Database Monitoring provides comprehensive visibility across multiple database engines, offering features like query-level performance tracking, execution plan analysis, and automated anomaly detection. The platform excels at correlating database performance with application metrics, helping teams understand the full impact of database issues on user experience.

SolarWinds Database Performance Analyzer takes a different approach, focusing on wait time analysis to identify performance bottlenecks. By examining what queries are waiting for and why, it helps database administrators understand resource contention and optimize accordingly. The platform's strength lies in its ability to provide historical context, allowing teams to identify performance trends and capacity planning needs.

Percona Monitoring and Management represents the open-source approach to database observability, offering deep insights into MySQL, PostgreSQL, and MongoDB environments. Its strength lies in providing detailed query analytics and performance schema integration, making it particularly valuable for organizations with complex, high-traffic database environments.

Traditional Databases Embracing Observability

Recognizing the critical importance of observability, traditional database vendors have integrated sophisticated monitoring capabilities directly into their platforms. Oracle's Autonomous Database includes built-in machine learning algorithms that continuously monitor performance patterns and automatically optimize configurations. This self-tuning capability represents a significant evolution from reactive monitoring to proactive performance management.

Microsoft SQL Server's Query Store functionality exemplifies how traditional databases are incorporating observability principles. By automatically capturing query execution statistics and maintaining historical performance data, SQL Server enables administrators to identify performance regressions and understand the impact of schema changes over time. The platform's integration with Azure Monitor further extends these capabilities into cloud environments.

PostgreSQL has enhanced its observability through extensions like pg_stat_statements and pg_stat_activity, which provide detailed insights into query performance and system activity. These built-in tools, combined with third-party solutions, create a comprehensive observability ecosystem that rivals dedicated monitoring platforms.

Navicat Monitor: Providing Comprehensive Database Insights

Navicat Monitor exemplifies the evolution of database observability tools by providing deep insights into database behavior, query performance, and resource utilization across multiple database types. The platform's strength lies in its ability to monitor heterogeneous database environments from a single interface, supporting MySQL, MariaDB, PostgreSQL, SQL Server, as well as popular cloud services.

The platform's real-time monitoring capabilities extend beyond basic performance metrics to include detailed query analysis, connection monitoring, and resource utilization tracking. Navicat Monitor's alerting system enables proactive issue resolution by notifying administrators of performance anomalies before they impact end users. Its historical reporting features provide valuable insights for capacity planning and performance trend analysis, making it an essential tool for organizations managing complex database infrastructures.

Conclusion

Database observability platforms represent a critical evolution in database management, transforming reactive monitoring into proactive performance optimization. As organizations continue to rely on increasingly complex data architectures, these platforms provide the visibility and insights necessary to maintain optimal performance while ensuring reliable data access. The integration of observability features into traditional database platforms, combined with specialized monitoring solutions, creates a comprehensive foundation that empowers database administrators to deliver exceptional performance and reliability.

In an era where data breaches make headlines weekly and privacy regulations like GDPR (General Data Protection Regulation) reshape how organizations handle personal information, privacy-preserving databases have emerged as a critical technology. These specialized database systems allow organizations to store, query, and analyze sensitive data while maintaining strict privacy protections for individuals whose information is contained within. This article explores the core technologies that make privacy protection possible, examines leading database solutions in this space, and discusses how both traditional database vendors and modern administration tools are adapting to support these privacy-first approaches.

Core Technologies Behind Privacy Protection

Privacy-preserving databases incorporate several key features that distinguish them from traditional database systems. Think of these features as multiple layers of protection, each serving a specific purpose in safeguarding sensitive information.

The foundation of these systems rests on differential privacy, a mathematical framework that adds carefully calibrated noise to query results. This approach ensures that whether any individual's data is included in the database or not, the statistical outputs remain virtually indistinguishable. Imagine trying to determine if a specific person attended a large concert by looking at aggregate attendance statisticsdifferential privacy makes this type of inference nearly impossible.

Homomorphic encryption represents another cornerstone feature, allowing computations to be performed directly on encrypted data without ever decrypting it. This means database queries can execute and return meaningful results while the underlying sensitive data remains encrypted throughout the entire process. It's analogous to performing mathematical operations inside a locked box without ever opening it.

Secure multi-party computation enables multiple parties to jointly compute functions over their combined data without revealing their individual inputs to each other. For instance, multiple hospitals could collaborate on medical research by combining their patient data for analysis without any hospital seeing another's specific patient records.

Zero-knowledge proofs allow database systems to verify the truth of statements about data without revealing the underlying information itself. These proofs can confirm that certain conditions are met or that specific computations were performed correctly without exposing the sensitive data involved.

Some Examples of Leading Privacy-Preserving Databases

Several innovative database systems have emerged to address these privacy challenges. CryptDB pioneered the field by enabling SQL queries over encrypted data, using multiple encryption schemes to support different types of database operations while maintaining security.

Opaque takes a different approach by combining hardware-based trusted execution environments with differential privacy. This system runs database queries inside secure enclaves that isolate computation from the underlying operating system and hardware, providing both confidentiality and integrity guarantees.

PrivateSQL focuses specifically on supporting complex analytical queries while preserving privacy through advanced cryptographic techniques. The system demonstrates how organizations can perform sophisticated data analysis without compromising individual privacy.

Microsoft's SEAL (Simple Encrypted Arithmetic Library) provides the cryptographic foundation for many privacy-preserving database implementations, offering homomorphic encryption capabilities that enable computation on encrypted data.

Traditional Databases Embracing Privacy Features

Established database vendors have recognized the growing demand for privacy protection and are integrating these capabilities into their existing platforms. This evolution represents a significant shift in how traditional database systems approach data protection.

PostgreSQL has incorporated extensions for differential privacy through projects like PostgreSQL Anonymizer, which provides tools for data masking and anonymization directly within the database engine. These features allow organizations to create privacy-safe versions of their datasets for testing and development purposes.

Oracle Database has introduced comprehensive data redaction and masking capabilities that can dynamically alter sensitive data presentation based on user privileges and context. The system can automatically detect and protect sensitive data types like credit card numbers and social security numbers.

Microsoft SQL Server has integrated Always Encrypted technology, which ensures that sensitive data remains encrypted at rest, in transit, and even during query processing. The database engine never sees the plaintext data, yet can still perform certain types of queries and operations.

Amazon's Aurora and other cloud database services now offer client-side encryption and key management services that enable organizations to maintain control over their encryption keys while leveraging cloud database capabilities.

Navicat's Role in Privacy-Preserving Database Management

Navicat's comprehensive database administration and development tools have evolved to support the unique requirements of privacy-preserving database environments. These tools recognize that managing encrypted or privacy-protected data requires specialized capabilities beyond traditional database administration.

The platform provides secure connection management that supports advanced encryption protocols and authentication mechanisms required by privacy-preserving systems. Database administrators can establish connections to encrypted databases while maintaining the security protocols that these systems demand.

Navicat's query development environment includes features for working with encrypted data and privacy-preserving query patterns. The tools help developers understand how their queries will interact with privacy protection mechanisms, enabling them to write more efficient and privacy-compliant database operations.

Conclusion

Privacy-preserving databases represent a fundamental shift in how we approach data management in an increasingly privacy-conscious world. By incorporating advanced cryptographic techniques and privacy-preserving algorithms, these systems enable organizations to derive value from sensitive data while maintaining robust protection for individual privacy. As traditional database vendors continue to integrate these capabilities and specialized tools like Navicat evolve to support them, privacy-preserving databases are becoming more accessible and practical for mainstream adoption. The future of data management lies not in choosing between utility and privacy, but in systems that provide both simultaneously through innovative technological approaches.

In an era where data breaches make headlines weekly and privacy regulations like GDPR (General Data Protection Regulation) reshape how organizations handle personal information, privacy-preserving databases have emerged as a critical technology. These specialized database systems allow organizations to store, query, and analyze sensitive data while maintaining strict privacy protections for individuals whose information is contained within. This article explores the core technologies that make privacy protection possible, examines leading database solutions in this space, and discusses how both traditional database vendors and modern administration tools are adapting to support these privacy-first approaches.

Core Technologies Behind Privacy Protection

Privacy-preserving databases incorporate several key features that distinguish them from traditional database systems. Think of these features as multiple layers of protection, each serving a specific purpose in safeguarding sensitive information.

The foundation of these systems rests on differential privacy, a mathematical framework that adds carefully calibrated noise to query results. This approach ensures that whether any individual's data is included in the database or not, the statistical outputs remain virtually indistinguishable. Imagine trying to determine if a specific person attended a large concert by looking at aggregate attendance statisticsdifferential privacy makes this type of inference nearly impossible.

Homomorphic encryption represents another cornerstone feature, allowing computations to be performed directly on encrypted data without ever decrypting it. This means database queries can execute and return meaningful results while the underlying sensitive data remains encrypted throughout the entire process. It's analogous to performing mathematical operations inside a locked box without ever opening it.

Secure multi-party computation enables multiple parties to jointly compute functions over their combined data without revealing their individual inputs to each other. For instance, multiple hospitals could collaborate on medical research by combining their patient data for analysis without any hospital seeing another's specific patient records.

Zero-knowledge proofs allow database systems to verify the truth of statements about data without revealing the underlying information itself. These proofs can confirm that certain conditions are met or that specific computations were performed correctly without exposing the sensitive data involved.

Some Examples of Leading Privacy-Preserving Databases

Several innovative database systems have emerged to address these privacy challenges. CryptDB pioneered the field by enabling SQL queries over encrypted data, using multiple encryption schemes to support different types of database operations while maintaining security.

Opaque takes a different approach by combining hardware-based trusted execution environments with differential privacy. This system runs database queries inside secure enclaves that isolate computation from the underlying operating system and hardware, providing both confidentiality and integrity guarantees.

PrivateSQL focuses specifically on supporting complex analytical queries while preserving privacy through advanced cryptographic techniques. The system demonstrates how organizations can perform sophisticated data analysis without compromising individual privacy.

Microsoft's SEAL (Simple Encrypted Arithmetic Library) provides the cryptographic foundation for many privacy-preserving database implementations, offering homomorphic encryption capabilities that enable computation on encrypted data.

Traditional Databases Embracing Privacy Features

Established database vendors have recognized the growing demand for privacy protection and are integrating these capabilities into their existing platforms. This evolution represents a significant shift in how traditional database systems approach data protection.

PostgreSQL has incorporated extensions for differential privacy through projects like PostgreSQL Anonymizer, which provides tools for data masking and anonymization directly within the database engine. These features allow organizations to create privacy-safe versions of their datasets for testing and development purposes.

Oracle Database has introduced comprehensive data redaction and masking capabilities that can dynamically alter sensitive data presentation based on user privileges and context. The system can automatically detect and protect sensitive data types like credit card numbers and social security numbers.

Microsoft SQL Server has integrated Always Encrypted technology, which ensures that sensitive data remains encrypted at rest, in transit, and even during query processing. The database engine never sees the plaintext data, yet can still perform certain types of queries and operations.

Amazon's Aurora and other cloud database services now offer client-side encryption and key management services that enable organizations to maintain control over their encryption keys while leveraging cloud database capabilities.

Navicat's Role in Privacy-Preserving Database Management

Navicat's comprehensive database administration and development tools have evolved to support the unique requirements of privacy-preserving database environments. These tools recognize that managing encrypted or privacy-protected data requires specialized capabilities beyond traditional database administration.

The platform provides secure connection management that supports advanced encryption protocols and authentication mechanisms required by privacy-preserving systems. Database administrators can establish connections to encrypted databases while maintaining the security protocols that these systems demand.

Navicat's query development environment includes features for working with encrypted data and privacy-preserving query patterns. The tools help developers understand how their queries will interact with privacy protection mechanisms, enabling them to write more efficient and privacy-compliant database operations.

Conclusion

Privacy-preserving databases represent a fundamental shift in how we approach data management in an increasingly privacy-conscious world. By incorporating advanced cryptographic techniques and privacy-preserving algorithms, these systems enable organizations to derive value from sensitive data while maintaining robust protection for individual privacy. As traditional database vendors continue to integrate these capabilities and specialized tools like Navicat evolve to support them, privacy-preserving databases are becoming more accessible and practical for mainstream adoption. The future of data management lies not in choosing between utility and privacy, but in systems that provide both simultaneously through innovative technological approaches.

In an era where data breaches make headlines weekly and privacy regulations like GDPR (General Data Protection Regulation) reshape how organizations handle personal information, privacy-preserving databases have emerged as a critical technology. These specialized database systems allow organizations to store, query, and analyze sensitive data while maintaining strict privacy protections for individuals whose information is contained within. This article explores the core technologies that make privacy protection possible, examines leading database solutions in this space, and discusses how both traditional database vendors and modern administration tools are adapting to support these privacy-first approaches.

Core Technologies Behind Privacy Protection

Privacy-preserving databases incorporate several key features that distinguish them from traditional database systems. Think of these features as multiple layers of protection, each serving a specific purpose in safeguarding sensitive information.

The foundation of these systems rests on differential privacy, a mathematical framework that adds carefully calibrated noise to query results. This approach ensures that whether any individual's data is included in the database or not, the statistical outputs remain virtually indistinguishable. Imagine trying to determine if a specific person attended a large concert by looking at aggregate attendance statisticsdifferential privacy makes this type of inference nearly impossible.

Homomorphic encryption represents another cornerstone feature, allowing computations to be performed directly on encrypted data without ever decrypting it. This means database queries can execute and return meaningful results while the underlying sensitive data remains encrypted throughout the entire process. It's analogous to performing mathematical operations inside a locked box without ever opening it.

Secure multi-party computation enables multiple parties to jointly compute functions over their combined data without revealing their individual inputs to each other. For instance, multiple hospitals could collaborate on medical research by combining their patient data for analysis without any hospital seeing another's specific patient records.

Zero-knowledge proofs allow database systems to verify the truth of statements about data without revealing the underlying information itself. These proofs can confirm that certain conditions are met or that specific computations were performed correctly without exposing the sensitive data involved.

Some Examples of Leading Privacy-Preserving Databases

Several innovative database systems have emerged to address these privacy challenges. CryptDB pioneered the field by enabling SQL queries over encrypted data, using multiple encryption schemes to support different types of database operations while maintaining security.

Opaque takes a different approach by combining hardware-based trusted execution environments with differential privacy. This system runs database queries inside secure enclaves that isolate computation from the underlying operating system and hardware, providing both confidentiality and integrity guarantees.

PrivateSQL focuses specifically on supporting complex analytical queries while preserving privacy through advanced cryptographic techniques. The system demonstrates how organizations can perform sophisticated data analysis without compromising individual privacy.

Microsoft's SEAL (Simple Encrypted Arithmetic Library) provides the cryptographic foundation for many privacy-preserving database implementations, offering homomorphic encryption capabilities that enable computation on encrypted data.

Traditional Databases Embracing Privacy Features

Established database vendors have recognized the growing demand for privacy protection and are integrating these capabilities into their existing platforms. This evolution represents a significant shift in how traditional database systems approach data protection.

PostgreSQL has incorporated extensions for differential privacy through projects like PostgreSQL Anonymizer, which provides tools for data masking and anonymization directly within the database engine. These features allow organizations to create privacy-safe versions of their datasets for testing and development purposes.

Oracle Database has introduced comprehensive data redaction and masking capabilities that can dynamically alter sensitive data presentation based on user privileges and context. The system can automatically detect and protect sensitive data types like credit card numbers and social security numbers.

Microsoft SQL Server has integrated Always Encrypted technology, which ensures that sensitive data remains encrypted at rest, in transit, and even during query processing. The database engine never sees the plaintext data, yet can still perform certain types of queries and operations.

Amazon's Aurora and other cloud database services now offer client-side encryption and key management services that enable organizations to maintain control over their encryption keys while leveraging cloud database capabilities.

Navicat's Role in Privacy-Preserving Database Management

Navicat's comprehensive database administration and development tools have evolved to support the unique requirements of privacy-preserving database environments. These tools recognize that managing encrypted or privacy-protected data requires specialized capabilities beyond traditional database administration.

The platform provides secure connection management that supports advanced encryption protocols and authentication mechanisms required by privacy-preserving systems. Database administrators can establish connections to encrypted databases while maintaining the security protocols that these systems demand.

Navicat's query development environment includes features for working with encrypted data and privacy-preserving query patterns. The tools help developers understand how their queries will interact with privacy protection mechanisms, enabling them to write more efficient and privacy-compliant database operations.

Conclusion

Privacy-preserving databases represent a fundamental shift in how we approach data management in an increasingly privacy-conscious world. By incorporating advanced cryptographic techniques and privacy-preserving algorithms, these systems enable organizations to derive value from sensitive data while maintaining robust protection for individual privacy. As traditional database vendors continue to integrate these capabilities and specialized tools like Navicat evolve to support them, privacy-preserving databases are becoming more accessible and practical for mainstream adoption. The future of data management lies not in choosing between utility and privacy, but in systems that provide both simultaneously through innovative technological approaches.

Database sharding represents one of the most powerful techniques for scaling databases horizontally, addressing the limitations that arise when a single database server can no longer handle the growing demands of modern applications. To understand sharding, imagine a massive library that has grown so large that patrons struggle to find books quickly. Rather than building a taller building, the librarians decide to create multiple smaller libraries, each specializing in certain subjects or alphabetical ranges. This distribution approach mirrors exactly what database sharding accomplishes.

At its core, sharding involves partitioning a large database into smaller, more manageable pieces called shards, with each shard residing on a separate server or cluster. Each shard contains a subset of the total data, typically divided based on a specific criterion such as customer ID ranges, geographical regions, or alphabetical sorting. This horizontal partitioning strategy differs fundamentally from vertical scaling, where you simply add more power to a single server, because it distributes both the data storage burden and the processing load across multiple systems.

The beauty of sharding lies in its ability to maintain performance as your application grows. When a single database server reaches its limits in terms of storage capacity, memory, or processing power, sharding allows you to add more servers to handle the increased load, rather than trying to upgrade to an impossibly powerful single machine. This article explores how Database Sharding as a Service has revolutionized horizontal database scaling by providing managed solutions that automatically distribute data across multiple servers, enabling organizations to achieve high-performance scalability without the traditional complexity of building and maintaining sharding infrastructure themselves.

A Quick History

Traditionally, implementing database sharding required significant technical expertise and substantial infrastructure management overhead. Database administrators needed to design sharding strategies, manage data distribution logic, handle cross-shard queries, and maintain consistency across multiple database instances. This complexity often made sharding accessible only to organizations with substantial technical resources and expertise.

Database Sharding as a Service has emerged as a game-changing solution that abstracts away much of this complexity. These services provide managed sharding solutions where the service provider handles the intricate details of shard management, data distribution, query routing, and infrastructure maintenance. This approach allows organizations to benefit from sharding's scalability advantages without needing to build and maintain the underlying sharding infrastructure themselves.

The service model transforms sharding from a complex technical challenge into a configurable feature. Organizations can focus on their core business logic while the service provider ensures optimal data distribution, handles failover scenarios, manages shard rebalancing, and maintains overall system performance.

Leading Database Sharding Services in the Market

Several prominent cloud providers and specialized database companies now offer sophisticated sharding services:

• Amazon Web Services provides sharding capabilities through Amazon RDS with read replicas and Amazon Aurora's distributed architecture, while their DynamoDB offers automatic partitioning that essentially provides sharding functionality without requiring manual configuration.
• Google Cloud offers sharding through Cloud Spanner, which automatically distributes data across multiple servers and regions while maintaining strong consistency guarantees. This service exemplifies how modern sharding solutions can handle complex distributed database challenges transparently.
• MongoDB Atlas represents another significant player in this space, providing automated sharding that can dynamically redistribute data as your application's needs change. The service monitors shard utilization and can automatically split or merge shards to maintain optimal performance.
• Microsoft Azure's Cosmos DB offers partitioning capabilities that function similarly to sharding, automatically distributing data across multiple physical partitions based on partition key strategies that developers define.

These services demonstrate how the industry has evolved to provide sharding capabilities that were once available only to companies with extensive database expertise and infrastructure resources.

How Navicat Simplifies Database Sharding Management

Working with sharded databases, whether through managed services or custom implementations, presents unique challenges for database administrators and developers. Navicat's comprehensive database administration and development tools provide essential capabilities that significantly streamline the management of sharded database environments.

Navicat's multi-database connectivity features allow administrators to establish connections to multiple shards simultaneously, providing a unified interface for managing distributed data. This capability proves invaluable when you need to execute administrative tasks across multiple database instances or when troubleshooting issues that span multiple shards.

The visual query builder and SQL editor in Navicat help developers construct and test queries that work effectively within sharded environments. Understanding how queries will perform across different shards becomes crucial for maintaining application performance, and Navicat's tools provide the visibility needed to optimize these distributed queries.

Additionally, Navicat's data synchronization and comparison tools become particularly valuable in sharded environments where maintaining data consistency and performing migrations between shards requires careful coordination. These tools help ensure that data remains properly distributed and synchronized across the sharded infrastructure.

Conclusion

Database Sharding as a Service represents a significant advancement in making horizontal database scaling accessible to organizations of all sizes. By abstracting the complexity of shard management while providing the performance benefits of distributed data storage, these services enable businesses to focus on growth rather than infrastructure challenges. As applications continue to generate ever-increasing amounts of data, understanding and leveraging these sharding services, supported by comprehensive database management tools like Navicat, becomes essential for maintaining competitive advantage in our data-driven world.

Database administrators and developers looking for a comprehensive database management solution will find plenty to appreciate in Navicat 17.3, the latest iteration of PremiumSoft's flagship database tool. This update represents a significant leap forward, particularly in artificial intelligence integration and database connectivity options, positioning Navicat as a forward-thinking solution in an increasingly competitive market.

Expanded Database Universe

The most immediately practical enhancement in version 17.3 is the expanded roster of supported database connections. The addition of five new connection types demonstrates Navicat's commitment to supporting diverse database ecosystems. Fujitsu Enterprise Postgres support acknowledges the growing enterprise adoption of PostgreSQL variants, while Azure Cosmos DB for MongoDB integration reflects the reality of hybrid cloud deployments that many organizations face today.

The inclusion of database systems like Dameng and KingBaseES is particularly noteworthy, suggesting Navicat's awareness of the global database landscape and the increasing importance of domestic database solutions in various markets. IvorySQL support further strengthens the PostgreSQL coverage, giving users more flexibility when working with PostgreSQL-compatible databases.

This expanded connectivity transforms Navicat from a multi-database tool into a truly universal database management platform, reducing the need for multiple specialized tools in complex environments.

AI Integration: The Game Changer

Where Navicat 17.3 truly distinguishes itself is in its comprehensive AI integration. The support for a range of AI models, including ChatGPT, Deepseek, Google Gemini, Ollama - and the newly added Grok, Claude, and Qwen - is a game changer, demonstrating Navicat's understanding that different AI assistants excel in different scenarios. This multi-model approach allows users to leverage the unique strengths of each platform rather than being locked into a single AI ecosystem.

The "Ask AI" feature with pinnable favorite actions represents thoughtful user experience design. Rather than forcing users to repeatedly access AI features through complex menus, the ability to pin frequently used AI actions creates personalized workflows that can significantly accelerate daily tasks. This is particularly valuable for SQL optimization and code quality improvements, where iterative refinement is common.

The "Compare with" functionality addresses a real pain point in AI-assisted development: the uncertainty about whether you're getting the best possible solution. By allowing direct comparison between different AI assistants' responses, users can make more informed decisions about their database queries and structures.

The "Fix with AI" and "Explain SQL" features tackle two of the most time-consuming aspects of database work. Automated error resolution suggestions can dramatically reduce debugging time, while SQL explanation capabilities serve as both a learning tool for junior developers and a documentation aid for complex queries.

User Interface Refinements

The user interface improvements, while more subtle than the AI enhancements, address practical daily workflow issues. Color-coded text comparison makes it significantly easier to spot differences in database schemas, reducing eye strain and potential errors during data analysis.

The improved code completion with "great-grandparent cases" support specifically addresses complex database relationships, where traditional autocomplete often falls short. The expandable Information Pane provides better workspace management, allowing users to maximize screen real estate when needed while keeping essential information accessible.

The Verdict

Navicat 17.3 represents a mature evolution of database management software, successfully balancing traditional database administration needs with cutting-edge AI capabilities. The expanded database support ensures broad compatibility, while the thoughtful AI integration provides genuine productivity improvements rather than superficial feature additions.

For database professionals working in multi-platform environments or those looking to leverage AI assistance in their daily workflows, Navicat 17.3 offers compelling value. The combination of universal database connectivity and intelligent assistance positions it as a tool that can grow with evolving database landscapes and development practices.

Navicat 17.3 is available for download today. For more information about the new features visit the Navicat 17 Highlights page. You'll also find link to try or buy Navicat 17.3 at the bottom of the page.

As organizations worldwide grapple with mounting environmental challenges, the technology sector faces increasing pressure to reduce its carbon footprint. Data centers alone consume approximately 1% of global electricity, making database efficiency a critical component of corporate sustainability strategies. Sustainability-focused databases represent a paradigm shift from traditional performance-only metrics to encompass environmental impact, energy efficiency, and resource optimization alongside conventional database capabilities.

This article explores how sustainability-focused databases represent a fundamental shift in data management philosophy, balancing traditional performance metrics with environmental considerations like energy efficiency and resource optimization to help organizations reduce their carbon footprint while maintaining reliable data operations. The key insight here is that this isn't simply about making existing databases use less power - it's about rethinking the entire approach to database design from the ground up. Just as hybrid cars required engineers to reconsider the fundamental relationship between power and efficiency, sustainable databases require us to view computational performance through an environmental lens, creating systems that are both effective and environmentally responsible.

The Green Revolution in Data Architecture

Sustainability-focused databases prioritize environmental considerations throughout their architecture and operation. Unlike conventional databases that optimize primarily for speed and reliability, these systems balance performance with energy consumption, hardware longevity, and resource utilization. The core principle involves minimizing computational overhead while maintaining data integrity and accessibility.

Think of this approach like designing a hybrid car versus a traditional vehicle. While both need to transport passengers efficiently, the hybrid considers fuel consumption and emissions as equally important design constraints. Similarly, sustainable databases weigh energy costs against query performance, seeking optimal efficiency rather than maximum speed at any environmental cost.

These databases typically incorporate several key features: intelligent query optimization that reduces processing cycles, compression algorithms that minimize storage requirements, and adaptive scaling that adjusts resource allocation based on actual demand rather than peak capacity planning.

Environmental Impact and Energy Efficiency

The environmental implications of database operations extend far beyond immediate electricity consumption. Traditional databases often operate with significant overhead, maintaining multiple redundant processes and keeping servers at constant high-performance states regardless of actual workload demands.

Sustainability-focused systems address this through dynamic resource management. When query loads decrease during off-peak hours, these databases can scale down processing power, reduce memory allocation, and even power down unnecessary hardware components. This approach parallels how modern buildings use smart lighting systems that automatically adjust brightness based on occupancy and natural light levels.

Furthermore, these databases optimize data storage through advanced compression techniques and intelligent archiving strategies. By reducing the physical storage footprint, organizations decrease their need for additional hardware, thereby reducing manufacturing-related emissions and extending the operational lifespan of existing infrastructure.

Implementation Strategies and Best Practices

Implementing sustainability-focused databases requires a comprehensive approach that considers both technical architecture and operational procedures. Organizations should begin by conducting energy audits of their existing database infrastructure to establish baseline consumption metrics and identify optimization opportunities.

The implementation process typically involves migrating to database systems that support dynamic scaling, implementing intelligent caching mechanisms to reduce redundant queries, and establishing data lifecycle management policies that automatically archive or compress older information. Companies should also consider geographic factors, such as locating data centers in regions with abundant renewable energy sources.

Successful implementation also requires staff training and cultural adaptation. Database administrators need to understand how traditional performance tuning techniques may conflict with sustainability goals, and development teams must learn to write queries that balance speed with resource efficiency. This educational component often proves as crucial as the technical migration itself.

Database Management Tools and Administration

Professional database administration tools play a vital role in successfully managing sustainability-focused databases. Navicat's comprehensive database administration and development platform provides essential capabilities for organizations transitioning to environmentally conscious data management. The suite offers advanced monitoring features that track both performance metrics and resource utilization patterns, enabling administrators to identify optimization opportunities and measure environmental impact improvements.

Navicat's tools facilitate efficient database design through visual modeling capabilities that help developers create optimized schemas from the outset. The platform's query optimization features assist in writing efficient SQL statements that minimize processing overhead, while its automated backup and maintenance scheduling reduces the need for energy-intensive manual interventions during peak usage periods.

Conclusion

Sustainability-focused databases represent more than an environmental initiative; they embody a fundamental rethinking of how we approach data management in a resource-constrained world. As regulatory pressures increase and stakeholder expectations evolve, organizations that proactively adopt sustainable database practices will find themselves better positioned for long-term success. The integration of environmental considerations into database design creates opportunities for cost reduction, operational efficiency, and competitive differentiation while contributing to broader climate goals. The transition requires careful planning and the right tools, but the benefits extend well beyond immediate environmental impact to encompass improved resource utilization and often enhanced system reliability.

The advent of quantum computing poses an unprecedented threat to traditional encryption methods that have secured our digital infrastructure for decades. Current cryptographic systems, including RSA, elliptic curve cryptography (ECC), and Diffie-Hellman key exchange, rely on mathematical problems that are computationally difficult for classical computers to solve. However, quantum computers running Shor's algorithm could theoretically break these encryption schemes in record time, rendering them virtually useless.

This threat isn't merely theoretical. Major technology companies and governments are investing billions in quantum computing research, with IBM, Google, and others achieving significant quantum milestones. While large-scale, fault-tolerant quantum computers capable of breaking current encryption may still be years away, the "Y2Q" (Years to Quantum) countdown has already begun. Organizations must prepare now, as encrypted data stolen today could be decrypted once quantum computers maturea concept known as "harvest now, decrypt later" attacks. This article explains how quantum computing threatens current encryption methods and how modern databases are implementing quantum-resistant encryption algorithms to protect data from future quantum computer attacks.

Guarding Agaist the Quantum Threat with Quantum-Resistant Encryption

Quantum-resistant encryption, also called post-quantum cryptography (PQC), represents a new class of cryptographic algorithms designed to withstand attacks from both classical and quantum computers. Unlike current methods based on integer factorization or discrete logarithms, quantum-resistant algorithms rely on mathematical problems that remain difficult even for quantum computers.

The National Institute of Standards and Technology (NIST) has been leading the standardization effort, selecting several algorithms after rigorous evaluation. Key approaches include lattice-based cryptography (CRYSTALS-Kyber for key encapsulation, CRYSTALS-Dilithium for digital signatures), hash-based signatures (SPHINCS+), and code-based cryptography. These algorithms offer varying trade-offs between security, performance, and key sizes, allowing organizations to choose appropriate solutions for their specific needs.

Modern Database Support for Quantum-Resistant Encryption

Database vendors are proactively implementing quantum-resistant encryption to protect sensitive data. IBM DB2 has integrated CRYSTALS-Kyber and CRYSTALS-Dilithium algorithms, providing quantum-safe key exchange and digital signatures. Oracle Database has added post-quantum cryptography support in recent versions, focusing on protecting data at rest and in transit.

Microsoft SQL Server now supports NIST-approved quantum-safe algorithms, while PostgreSQL offers extensions for post-quantum encryption capabilities. Cloud database providers are also advancing quantum readinessAmazon RDS and Aurora participate in AWS's quantum-safe cryptography initiatives, Google Cloud SQL supports post-quantum TLS protocols, and Azure SQL Database implements Microsoft's quantum-resistant solutions.

Specialized databases like CockroachDB have built-in quantum-resistant algorithm support, while MongoDB Atlas and Apple's FoundationDB offer post-quantum encryption options. These implementations typically focus on three critical areas: encrypting data at rest, securing data in transit through quantum-safe TLS, and protecting authentication processes with quantum-resistant digital signatures.

Navicat: Secure Database Administration in the Quantum Era

As organizations transition to quantum-resistant encryption, reliable database administration tools become crucial for managing security implementations effectively. Navicat's comprehensive database administration and development tools provide essential capabilities for working securely with modern databases. The platform supports secure connections across multiple database systems, enabling administrators to manage encrypted databases with confidence.

Navicat's tools facilitate secure database connections through advanced encryption protocols, helping database professionals implement and maintain security best practices. The platform's intuitive interface allows administrators to configure security settings, monitor database access, and ensure compliance with evolving cryptographic standards without compromising productivity or functionality.

Conclusion

The transition to quantum-resistant encryption represents one of the most significant security upgrades in computing history. As quantum computing advances, organizations cannot afford to waitthe time for preparation is now. Modern database systems are already implementing post-quantum cryptography, providing the foundation for long-term data security.

Success in this transition requires not only adopting quantum-resistant algorithms but also utilizing professional-grade database administration tools that support secure implementation and management. By combining quantum-safe encryption with robust database management practices, organizations can build resilient data infrastructure ready for the quantum future.

The advent of quantum computing poses an unprecedented threat to traditional encryption methods that have secured our digital infrastructure for decades. Current cryptographic systems, including RSA, elliptic curve cryptography (ECC), and Diffie-Hellman key exchange, rely on mathematical problems that are computationally difficult for classical computers to solve. However, quantum computers running Shor's algorithm could theoretically break these encryption schemes in record time, rendering them virtually useless.

This threat isn't merely theoretical. Major technology companies and governments are investing billions in quantum computing research, with IBM, Google, and others achieving significant quantum milestones. While large-scale, fault-tolerant quantum computers capable of breaking current encryption may still be years away, the "Y2Q" (Years to Quantum) countdown has already begun. Organizations must prepare now, as encrypted data stolen today could be decrypted once quantum computers maturea concept known as "harvest now, decrypt later" attacks. This article explains how quantum computing threatens current encryption methods and how modern databases are implementing quantum-resistant encryption algorithms to protect data from future quantum computer attacks.

Guarding Agaist the Quantum Threat with Quantum-Resistant Encryption

Quantum-resistant encryption, also called post-quantum cryptography (PQC), represents a new class of cryptographic algorithms designed to withstand attacks from both classical and quantum computers. Unlike current methods based on integer factorization or discrete logarithms, quantum-resistant algorithms rely on mathematical problems that remain difficult even for quantum computers.

The National Institute of Standards and Technology (NIST) has been leading the standardization effort, selecting several algorithms after rigorous evaluation. Key approaches include lattice-based cryptography (CRYSTALS-Kyber for key encapsulation, CRYSTALS-Dilithium for digital signatures), hash-based signatures (SPHINCS+), and code-based cryptography. These algorithms offer varying trade-offs between security, performance, and key sizes, allowing organizations to choose appropriate solutions for their specific needs.

Modern Database Support for Quantum-Resistant Encryption

Database vendors are proactively implementing quantum-resistant encryption to protect sensitive data. IBM DB2 has integrated CRYSTALS-Kyber and CRYSTALS-Dilithium algorithms, providing quantum-safe key exchange and digital signatures. Oracle Database has added post-quantum cryptography support in recent versions, focusing on protecting data at rest and in transit.

Microsoft SQL Server now supports NIST-approved quantum-safe algorithms, while PostgreSQL offers extensions for post-quantum encryption capabilities. Cloud database providers are also advancing quantum readinessAmazon RDS and Aurora participate in AWS's quantum-safe cryptography initiatives, Google Cloud SQL supports post-quantum TLS protocols, and Azure SQL Database implements Microsoft's quantum-resistant solutions.

Specialized databases like CockroachDB have built-in quantum-resistant algorithm support, while MongoDB Atlas and Apple's FoundationDB offer post-quantum encryption options. These implementations typically focus on three critical areas: encrypting data at rest, securing data in transit through quantum-safe TLS, and protecting authentication processes with quantum-resistant digital signatures.

Navicat: Secure Database Administration in the Quantum Era

As organizations transition to quantum-resistant encryption, reliable database administration tools become crucial for managing security implementations effectively. Navicat's comprehensive database administration and development tools provide essential capabilities for working securely with modern databases. The platform supports secure connections across multiple database systems, enabling administrators to manage encrypted databases with confidence.

Navicat's tools facilitate secure database connections through advanced encryption protocols, helping database professionals implement and maintain security best practices. The platform's intuitive interface allows administrators to configure security settings, monitor database access, and ensure compliance with evolving cryptographic standards without compromising productivity or functionality.

Conclusion

The transition to quantum-resistant encryption represents one of the most significant security upgrades in computing history. As quantum computing advances, organizations cannot afford to waitthe time for preparation is now. Modern database systems are already implementing post-quantum cryptography, providing the foundation for long-term data security.

Success in this transition requires not only adopting quantum-resistant algorithms but also utilizing professional-grade database administration tools that support secure implementation and management. By combining quantum-safe encryption with robust database management practices, organizations can build resilient data infrastructure ready for the quantum future.

Blockchain technology has rapidly evolved from its cryptocurrency origins to become a compelling data management system in its own right. Modern blockchain databases represent a significant advancement in how organizations approach data integrity, transparency, and security. These systems combine the benefits of distributed ledger technology with the functionality of traditional database management systems, creating hybrid solutions that address long-standing challenges in data governance. As enterprises increasingly seek solutions that provide immutable audit trails and verifiable transaction history, blockchain databases have emerged as a promising option that balances innovation with practical business requirements. This article describes how blockchain databases work and lists some of the most popular blockchain database solutions, along with some traditional alternatives implementing similar features. Finally, we'll examine how specialized tools like Navicat are helping organizations bridge these two worlds.

Understanding Blockchain Databases

Blockchain databases fundamentally differ from conventional databases in their architecture and operating principles. While traditional databases typically function as centralized repositories managed by a single authority, blockchain databases distribute data across multiple nodes in a network. Each transaction or data change is recorded in the ledger as a "block" that contains a cryptographic hash linking it to the previous block, creating an unalterable chain of information. This structure ensures that once data is recorded, it cannot be modified without consensus from the network, providing unprecedented levels of data integrity and auditability.

The core features that distinguish blockchain databases include immutability, decentralized consensus mechanisms, cryptographic verification, and transparent transaction history. These characteristics make blockchain databases particularly valuable for applications requiring robust audit trails, such as financial systems, supply chain management, and regulatory compliance.

Leading Blockchain Database Solutions

Several blockchain database platforms have gained prominence in the enterprise space. Here are just a few:

• BigchainDB combines the scalability of traditional distributed databases with blockchain features like immutability and decentralized control. It's designed for use cases requiring high throughput while maintaining blockchain's core benefits.
• Hyperledger Fabric, developed under the Linux Foundation, offers a permissioned blockchain framework specifically designed for enterprise use. It supports complex queries, private channels for sensitive data, and modular architecture that allows for customizable consensus mechanisms.
• Amazon QLDB (Quantum Ledger Database) provides a centrally managed ledger database with an immutable and cryptographically verifiable transaction log. Though not fully decentralized, it offers many blockchain benefits without the complexity of managing a distributed network.
• FlureeDB represents a new generation of blockchain databases, integrating graph database capabilities with blockchain features. This allows for complex data relationships while maintaining verifiable history and time-travel queries.
• Blockstore implements a decentralized key-value store using blockchain principles, making it suitable for applications requiring simple data structures with strong integrity guarantees.

Traditional Databases with Blockchain-Like Features

Traditional database vendors have recognized the value of blockchain's core principles and incorporated similar features into their products:

• PostgreSQL can implement append-only tables and cryptographic verification through extensions like pg_crypto, enabling some blockchain-like capabilities within a familiar relational environment.
• MongoDB offers change streams and immutable field features that provide auditing capabilities similar to blockchain ledgers, although without distributed consensus.
• Oracle Blockchain Tables extend standard Oracle database functionality with immutability guarantees and cryptographic verification, allowing organizations to maintain familiar SQL interfaces while gaining some blockchain benefits.
• Microsoft SQL Server Ledger introduces tamper-evidence features through cryptographic verification of historical data, addressing compliance and audit requirements within a traditional database framework.
• Immudb provides an open-source immutable database with cryptographic verification without the full overhead of blockchain, striking a balance between conventional database performance and blockchain integrity.

Database Administration with Navicat

For organizations implementing blockchain databases or blockchain-like features in traditional systems, effective database administration tools become essential. Navicat's suite of database management and development tools has evolved to support these advanced database technologies. Navicat provides intuitive interfaces for connecting to and managing both traditional databases with blockchain features and dedicated blockchain database systems.

Navicat's visual query builders and data visualization capabilities help developers and administrators work effectively with complex blockchain data structures. The tool's robust security features align well with blockchain's emphasis on data integrity, offering encrypted connections and comprehensive access controls. For teams implementing hybrid database architectures that combine blockchain and traditional elements, Navicat's support for multiple database types within a single interface streamlines workflow and reduces the learning curve associated with new technologies.

Conclusion

Blockchain databases represent a significant evolution in data management, introducing principles of immutability and distributed verification that address crucial gaps in traditional systems. The key distinction remains that true blockchain databases distribute trust across multiple parties through decentralized consensus, while traditional databases implementing blockchain-like features maintain centralized control while adding verification layers.

As organizations evaluate their data management strategies, the choice between pure blockchain databases, traditional systems with blockchain features, or hybrid approaches will depend on specific requirements for performance, scalability, compliance, and governance. With the support of advanced administration tools like Navicat, teams can effectively implement and manage these sophisticated database solutions, leveraging the best of both worlds to create robust, verifiable data systems suitable for today's complex business environments.

Introduction

The modern world is undergoing a significant transformation with the integration of Artificial Intelligence (AI) and Machine Learning (ML) capabilities into practically every facet of our lives. The emerging trend of embedded AI/ML functionality has now made its way into database systems, forever changing how organizations process, analyze, and derive value from their data assets. Rather than extracting data from databases to perform analytics in separate environments, these new systems enable real-time insights and predictions within the database itself, eliminating data movement and accelerating time-to-insight. This article will explore how the embedding of AI/ML capabilities directly into database systems enables real-time analytics, eliminates data movement challenges, and democratizes access to advanced predictive capabilities across organizations.

The Evolution of Database Intelligence

Traditional database systems have primarily served as repositories for structured data storage and retrieval. Over time, they evolved to incorporate more advanced analytical capabilities, but these were often limited to aggregations, statistical functions, and basic pattern recognition. The latest evolution brings sophisticated machine learning algorithms directly into the database engine, creating a unified platform for both data management and advanced analytics.

This convergence addresses a fundamental challenge in the data science workflow: the constant movement of data between storage systems and analytical environments. By embedding AI/ML capabilities within the database itself, organizations can dramatically reduce latency, enhance security, and improve governance while maintaining data freshness.

Key Capabilities and Benefits

Embedded AI/ML in databases offers several transformative capabilities. Automated feature engineering can identify relevant patterns and relationships within datasets, reducing the manual effort traditionally required from data scientists. Real-time anomaly detection can continuously monitor incoming data streams, immediately flagging unusual patterns that might indicate fraud, system failures, or business opportunities.

Predictive analytics functions allow users to create and deploy models using SQL-like syntax, democratizing access to sophisticated forecasting capabilities. These models can be trained on historical data and automatically updated as new information arrives, maintaining their accuracy over time without external intervention.

From an operational standpoint, the benefits are substantial. Processing data where it resides eliminates the security risks associated with data movement between systems. It also reduces infrastructure complexity and costs by consolidating what were previously separate systems for storage and analytics. The simplified architecture leads to better governance, as security policies, access controls, and audit trails can be managed in a single environment.

Leading Database Platforms Embracing AI/ML Integration

Major database vendors have recognized this trend and are rapidly enhancing their offerings. Microsoft SQL Server has introduced Machine Learning Services, enabling R and Python code execution within the database engine. Oracle's Autonomous Database incorporates machine learning algorithms for self-tuning, security, and predictive analytics. PostgreSQL extensions like MADlib provide scalable in-database machine learning algorithms through SQL interfaces.

Cloud-native databases have been particularly quick to adopt these capabilities. Amazon Redshift ML allows users to create, train, and deploy machine learning models using SQL commands. Google BigQuery ML similarly enables machine learning model building directly in the data warehouse using standard SQL syntax, while Snowflake's Snowpark brings data science workloads directly to where data resides.

Database Management Tools Incorporating AI

Database management tools are also incorporating AI technologies to enhance user experience and productivity. These tools leverage artificial intelligence to assist database administrators and developers with query optimization, schema design, and data management tasks. One notable example is Navicat's AI Assistant feature. Released in version 17.2, Navicat AI Assistant is an integrated tool that provides instant, contextual guidance and answers within a software application, leveraging artificial intelligence to help users solve problems, understand features, and improve their workflow through natural language interactions. Navicat's AI Assistant helps you write your SQL statements more efficiently. It does this by submitting your inquiries to the AI providers for processing, with responses sent exclusively back to the Navicat application installed on your local device. You can receive guidance from many of the popular AI chatbots, including ChatGPT, Google Gemini, DeepSeek, and Ollama.

Conclusion

The integration of AI/ML capabilities directly into database systems represents a natural evolution in data management technology. As organizations continue to grapple with exponentially growing data volumes and increasingly complex analytical requirements, embedded AI/ML functionality will become a standard feature rather than a differentiator.

This trend promises to democratize access to advanced analytics, allowing organizations of all sizes to derive actionable insights from their data assets without the complexity and expense of maintaining separate analytical infrastructures. As these technologies mature, we can expect even deeper integration between traditional database functions and cutting-edge AI/ML capabilities, further blurring the lines between data storage, management, and analysis.

In the evolving realm of database technology, immutable databases have emerged as a powerful new trend in data management that prioritizes data integrity and historical preservation. Unlike traditional databases where data can be modified or deleted, immutable databases only allow data addition, creating a permanent, tamper-proof record of all information. This article explores the rise of immutable databases and covers how database management tools like Navicat can help organizations effectively leverage these powerful capabilities.

The Concept of Immutability

Immutability in databases means that once data is written, it cannot be changed or deleted. Instead of updating or removing existing records, new versions are appended, preserving the complete history of changes. This append-only model ensures data integrity, simplifies auditing, and enables point-in-time recovery capabilities that traditional databases struggle to provide efficiently.

The immutable approach transforms how we think about data storage. Rather than maintaining the current state of data, immutable databases maintain the entire evolution of data over time. This shift brings significant advantages for compliance, security, and system reliability, particularly in industries where data provenance and auditability are critical.

Notable Immutable Database Examples

Several database systems have embraced immutability as their core design principle. Here are some of the main ones:

• Datomic pioneered the immutable database concept with its time-aware architecture. It stores all facts as datoms (atomic pieces of data) with time coordinates, allowing queries against any historical state without performance penalties.
• LMDB (Lightning Memory-Mapped Database) implements immutability through a copy-on-write mechanism, providing exceptional read performance and crash resilience.
• InfluxDB, primarily a time-series database, incorporates immutability for time-series data points, making it ideal for monitoring applications and systems where historical data must be preserved accurately.
• Event sourcing databases like EventStoreDB maintain an immutable log of all events, allowing systems to reconstruct state at any point in time by replaying events from the beginning or from snapshots.
• Blockchain databases like BigchainDB, Amazon Quantum Ledger Database (QLDB), and Hyperledger Fabric represent perhaps the most strict implementation of immutability, where the cryptographic linking of data blocks makes historical records practically impossible to alter without detection.

Traditional Databases Adopting Immutability

Recognizing the benefits of immutability, many traditional database systems have begun incorporating immutable features:

• PostgreSQL has implemented time travel capabilities through extensions like Temporal Tables and pg_audit, allowing developers to query data as it existed at previous points in time.
• Microsoft SQL Server introduced Temporal Tables in SQL Server 2016, providing built-in support for tracking historical data changes with period tables.
• Oracle database offers Flashback Query functionality, enabling users to view data as it existed at a specific time in the past without complex recovery procedures.
• MongoDB implemented Change Streams to provide applications with a real-time feed of data changes, preserving modification history in a way that mirrors some immutable database concepts.
• Amazon's DynamoDB offers Point-in-Time Recovery features that maintain a complete change history for tables, allowing restoration to any second in the previous 35 days.

Database Management with Navicat

When working with databases that incorporate immutable features, powerful database management tools become essential. Navicat stands out as a comprehensive solution that supports all major database systems implementing immutability concepts, including PostgreSQL, MySQL, MariaDB, SQL Server, Oracle, and MongoDB.

Navicat's intuitive interface allows database administrators to effectively manage temporal data and historical records created by immutable database features. Its visual query builder can construct complex queries against temporal tables, while its data modeling tools help design schemas that effectively incorporate immutability. For organizations transitioning to immutable data patterns, Navicat's synchronization and migration tools streamline the process of moving data between different database systems while preserving historical integrity.

Conclusion

Immutable databases represent a fundamental shift in how we store, process, and think about data. By prioritizing the preservation of history and guaranteeing data integrity, they provide solutions to many challenges faced in data management today. As traditional database systems continue to adopt immutability features, and purpose-built immutable databases mature, organizations gain powerful new tools for compliance, auditing, and system resilience. With proper management tools like Navicat, leveraging these capabilities becomes accessible even to teams without specialized knowledge of immutable data structures.

Modern enterprises face an unprecedented data management challenge. Organizations typically store their data across numerous systemscloud storage platforms, on-premises databases of various types, data warehouses, NoSQL repositories, SaaS applications, and specialized analytical systems. This data fragmentation creates significant obstacles for business users and analysts who need a comprehensive view of information to make decisions. Retrieving data from multiple systems requires mastering various query languages, understanding different data models, and manually integrating resultstasks too complex and time-consuming for most business users. The traditional solution of copying all data into a centralized repository creates its own problems: data duplication, staleness, increased storage costs, and complex synchronization processes. This article explores how data virtualization and federation technologies create a unified view of enterprise data scattered across disparate systems.

What is Data Virtualization and Federation?

Data virtualization represents a new approach to data integration that addresses these fundamental challenges. Rather than physically moving and consolidating data, data virtualization creates an abstraction layer that provides users and applications with unified, real-time access to data across disparate sources. This technology acts as a semantic layer that hides the technical complexities of underlying data systems, presenting a simplified view that users can interact with using familiar query tools and business intelligence interfaces. The virtualization engine translates user requests into source-specific queries, executes them across the relevant systems, and assembles the results into a coherent response - all while maintaining the illusion that users are working with a single, integrated data source.

Data federation functions as a fundamental architectural component within data virtualization solutions. Federation specifically addresses the mechanics of querying multiple heterogeneous data sources and combining their results. Federation engines decompose complex queries, determine which portions should be executed on which source systems, optimize these distributed query plans, and then reassemble the partial results. Modern federation technologies employ sophisticated optimization techniques, including pushing operations like filtering and aggregation down to source systems when possible, minimizing data transfer across networks, and caching frequently accessed data. Federation creates a virtual unified schema that maps fields from different systems into a coherent data model, handling complex transformations like field name standardization, data type conversion, and computational derivations.

Business Benefits of Virtualization and Federation

Implementing data virtualization and federation delivers several transformative business benefits. First, it dramatically accelerates time-to-insight by eliminating the need for physical data consolidation projects that often take months to complete. Business users gain immediate access to integrated views across systems, enabling faster decision-making. Second, these technologies reduce overall data management costs by minimizing unnecessary data replication and storage. Third, data virtualization enhances data governance by maintaining a single access point where security policies, data quality rules, and regulatory controls can be consistently applied. Perhaps most importantly, virtualization creates agilityas business requirements evolve, virtual views can be modified without disrupting the underlying systems or requiring extensive ETL modifications. This flexibility proves particularly valuable when integrating new data sources or adapting to organizational changes.

Implementation Considerations and Challenges

Successfully implementing data virtualization requires careful planning and awareness of potential challenges. Performance management represents the foremost concernfederated queries that span multiple systems inevitably introduce some latency compared to queries against a single optimized database. Organizations must develop strategies for managing this trade-off, such as implementing intelligent caching mechanisms, pre-aggregating commonly accessed data, or establishing clear performance expectations with users. Data security presents another critical consideration, as virtualization creates new access paths to sensitive information. Implementers must ensure that security controls remain consistent across the virtual layer and all underlying sources. Finally, organizations must recognize that virtualization complements rather than replaces other data integration approachessome use cases still benefit from physical consolidation, particularly those requiring historical analysis of large datasets or complex analytical processing.

Tools for Data Virtualization and Federation

Database management tools like Navicat can play a valuable supporting role in data virtualization and federation initiatives. While not a dedicated virtualization platform itself, Navicat provides capabilities that enhance the planning, implementation, and management phases of these projects. Its visual query builder allows database professionals to design and test complex federated queries across heterogeneous database environments. Navicat's schema comparison and synchronization features help maintain consistency across data sources that participate in federation schemas. The tool's support for multiple database typesincluding MySQL, PostgreSQL, SQL Server, Oracle, and MariaDBfacilitates the cross-platform data access essential to federation. Additionally, Navicat's data modeling capabilities assist in designing the unified semantic layer that makes virtualized data meaningful to business users, bridging the technical details of diverse sources with a coherent business-friendly representation.

Conclusion

Data virtualization and federation technologies represent a strategic approach to enterprise data integration challenges. By creating a unified access layer that preserves the underlying distribution of data, these technologies enable organizations to balance the competing demands of data consolidation and specialization. While implementing virtualization requires careful consideration of performance, security, and governance factors, the resulting benefitsfaster time-to-insight, reduced data management costs, and enhanced organizational agilitymake it an essential component of modern data architecture.

In traditional software development workflows, database changes have often been treated as an afterthought. While application code follows well-defined DevOps practices with version control, automated testing, and continuous deployment, database changes frequently remain manual, risky operations performed by database administrators. This disconnect creates bottlenecks, introduces errors, and slows down the entire development process. Organizations find themselves unable to deliver value quickly when database changes become the limiting factor in deployments. In this article, we'll explore how integrating database changes into DevOps workflows creates a more seamless development pipeline, examining the challenges, components, benefits, and implementation strategies of Database DevOps.

What is Database DevOps?

Database DevOps extends DevOps principles to database management, treating database code with the same rigor and automation as application code. It aims to bridge the gap between developers and database administrators by implementing consistent processes for database changes throughout the application lifecycle. The core philosophy is that database changes should be version-controlled, tested automatically, and deployed through reliable, repeatable processes - just like application code.

Key Components of Database DevOps

The successful implementation of Database DevOps relies on several interconnected components. First, all database objectstables, views, stored procedures, and functionsmust be represented as scripts in a version control system like Git. This provides a single source of truth for the database schema and enables collaboration between team members.

Second, continuous integration pipelines should automatically validate database changes. This includes syntax checking, running static analysis tools to identify potential performance issues, and executing tests against a test database to verify that changes won't break existing functionality.

Third, Database DevOps requires automated deployment tools that can apply changes to databases in different environments. These tools must handle complex scenarios like data migrations, schema changes, and rollbacks while preserving data integrity.

Finally, monitoring and observability tools complete the feedback loop by providing insights into database performance and potential issues, allowing teams to make informed decisions about future improvements.

Benefits of Database DevOps Integration

Organizations that successfully implement Database DevOps experience numerous benefits. Development cycles accelerate as database changes no longer create bottlenecks in the deployment process. The risk of production issues decreases thanks to thorough automated testing and consistent deployment processes. Compliance improves through comprehensive change tracking and auditing capabilities. Team collaboration strengthens when developers and database administrators work together using shared tools and processes. Perhaps most importantly, businesses can respond more quickly to market changes and customer needs when database changes can be deployed rapidly and reliably.

Implementation Strategies

Implementing Database DevOps requires a strategic approach. Start small by identifying a suitable project or database for a pilot implementation. Focus initially on version-controlling your database schemas and building basic validation tests. As your team gains confidence, expand to include more complex elements like stored procedures and functions.

Invest in training for both developers and database administrators to ensure everyone understands the new processes and tools. Create clear guidelines for database changes, including naming conventions, documentation requirements, and review processes.

Consider adopting a migration-based approach where each change is represented as a discrete migration script that can be applied in sequence. This approach makes it easier to track changes and perform rollbacks if necessary.

Tools for Database DevOps

Successful Database DevOps implementation relies on appropriate tooling. Database management systems like Navicat provide many essential capabilities that support DevOps practices for databases:

• Navicat offers schema comparison and synchronization features that help identify and deploy database changes systematically.
• Its data modeling capabilities serve as part of the database design process, while query building and optimization features improve code quality.
• Navicat's ability to generate SQL scripts that can be included in version control systems bridges the gap between database administration and development practices, making it a valuable component in a Database DevOps toolchain.

Conclusion

Database DevOps integration represents a significant evolution in how organizations manage database changes. By applying the same principles of automation, version control, and continuous testing that have transformed application development, teams can eliminate the database bottleneck and achieve truly continuous delivery of value to their customers. While challenges exist, particularly around cultural change and legacy systems, the benefits of faster delivery, reduced risk, and improved collaboration make Database DevOps a worthwhile investment for any organization that relies on databases to deliver value.

Were proud to announce that Navicat is a sponsor of SQLBits 2025, taking place in London from 1821 June. As the largest Data Platform conference in the world, SQLBits brings together over 200 leading experts, and a community committed to advancing the data industry.

Our sponsorship reflects our strong support for the ongoing evolution of modern data platforms and the professionals who build and manage them. With over 95% of the content being non-marketing driven, SQLBits offers invaluable real-world insights into todays most effective data strategies.

At Navicat, were dedicated to creating AI-powered database tools that enhance productivity, simplify management, and empower developers as part of a modern DevOps ecosystem. We remain committed to helping teams work smarter, faster, and more efficiently.

Learn more about the event at sqlbits.com

Edge computing has revolutionized how we process data by bringing computation closer to data sources. As organizations deploy more IoT devices, mobile applications, and distributed systems, the need for efficient edge database solutions has grown significantly. These specialized databases are designed to operate effectively on devices with limited processing power, memory, and network connectivity while ensuring data remains available and processable even when disconnected from central servers. Edge databases represent a fundamental shift in how we think about data architecture, enabling real-time processing and analytics where data is generated rather than requiring constant transmission to distant data centers. This article explores the emerging field of edge database solutions, examining how these specialized data management systems are designed to operate efficiently on devices with limited resources at the network periphery, comparing their unique benefits to traditional database approaches, and highlighting key technologies that enable local data processing and synchronization in disconnected or bandwidth-constrained environments.

What Are Edge Databases?

Edge databases are specialized data management systems optimized to run on edge devices such as smartphones, IoT sensors, retail terminals, manufacturing equipment, and other computing devices operating at the network periphery. Unlike traditional database systems that assume consistent connectivity and substantial computing resources, edge databases are engineered with different priorities. They're designed to be lightweight with minimal resource consumption, support offline operations, synchronize efficiently when connectivity is available, and provide reliable local data processing capabilities regardless of connection status.

These databases typically implement sophisticated data synchronization mechanisms that can resolve conflicts when devices reconnect after operating independently. They often employ intelligent data prioritization to ensure critical information is processed first when bandwidth is limited. The architecture of edge databases emphasizes fault tolerance and resilience, acknowledging the challenging and often unpredictable environments in which edge devices operate.

Benefits of Edge Database Solutions

Edge databases deliver several significant advantages over traditional centralized approaches:

• Latency reduction stands as perhaps the most immediate benefit - by processing data locally, applications can respond in real-time without waiting for round-trip communication with distant servers. This speed improvement proves crucial for time-sensitive applications like industrial control systems, autonomous vehicles, or medical devices where milliseconds matter.
• Privacy and security improve substantially as sensitive data can be processed locally without transmission across networks. This localized approach helps organizations comply with data sovereignty requirements and reduces overall vulnerability to network-based attacks.
• Bandwidth consumption decreases dramatically as only necessary data needs transmission to central systems rather than raw data streams. This efficiency translates directly to cost savings, particularly important in environments with metered or expensive connectivity.
• Reliability improves as applications continue functioning during network outages or in regions with inconsistent connectivity. This resilience ensures continuous operation in remote locations, developing regions, or crisis scenarios where network infrastructure may be compromised.

Popular Edge Database Solutions

Several database technologies have emerged specifically designed for edge computing scenarios:

• SQLite stands as perhaps the most widely deployed embedded database, powering countless applications across mobile devices and embedded systems. Its small footprint (approximately 600KB) and self-contained design make it ideal for edge deployments while still offering robust SQL capabilities.
• CouchDB and its mobile variant PouchDB provide powerful document-oriented databases with sophisticated synchronization mechanisms. Their multi-master replication allows multiple edge devices to operate independently and later reconcile changes seamlessly.
• RxDB combines reactive programming principles with offline-first architecture, making it particularly well-suited for progressive web applications and mobile scenarios. Its observable queries automatically update user interfaces when underlying data changes.
• Firebase Realtime Database offers real-time synchronization capabilities with offline support, simplifying development while handling complex networking challenges transparently.
• Berkeley DB provides a high-performance embedded database requiring minimal configuration while offering advanced features like transactions and recovery.

Edge Databases vs. Traditional Solutions

Traditional database systems like MySQL, PostgreSQL, and SQL Server were designed assuming consistent network connectivity, steady power supply, and substantial computing resources. These assumptions make them poorly suited for edge environments where intermittent connectivity and resource constraints are the norm.

Cloud database services like Amazon DynamoDB, Google Cloud Spanner, and Azure Cosmos DB offer powerful capabilities but generally require consistent connectivity to function properly. While these services increasingly offer offline capabilities, they still primarily operate under a centralized model.

Edge databases, in contrast, prioritize local operation first, with synchronization as a secondary concern. They employ sophisticated conflict resolution mechanisms that traditional databases often lack, handling the reality that multiple devices may independently modify the same data while disconnected.

Management Tools for Edge Databases

Managing distributed edge databases presents unique challenges compared to centralized systems. Administrators need visibility into device status, synchronization health, and data consistency across potentially thousands of endpoints. Navicat may be used to manage edge databases, offering tools to monitor synchronization status, troubleshoot replication conflicts, and ensure data integrity across distributed systems. Having proper management tools becomes essential as edge deployments scale to ensure system reliability and data consistency.

Conclusion

Edge database solutions represent a critical evolution in data management philosophy, recognizing that not all data processing must occur in centralized clouds. As edge computing continues expanding across industries, these specialized databases will play an increasingly vital role in enabling responsive, resilient applications that work reliably regardless of network conditions. Organizations implementing edge strategies should carefully evaluate database options based on their specific requirements for synchronization, offline capability, and resource efficiency to build truly effective distributed systems.

As the volume of data collected continues to increase at an exponential rate, the ability to effectively manage and analyze information has become critical across virtually every industry. Traditionally, working with databases required specialized technical skills, including proficiency in Structured Query Language (SQL) and database architecture principles. However, the emergence of low-code and no-code database interfaces is fundamentally transforming how organizations interact with their data assets. These innovative platforms empower business users, analysts, and even technical professionals to accomplish sophisticated database tasks with minimal manual coding, effectively democratizing access to data management capabilities while accelerating development cycles. This article explores how low-code and no-code database interfaces are transforming data management, examining their key benefits, organizational impact, and how tools like Navicat empower users to accomplish sophisticated database tasks with minimal manual coding.

Low-Code versus No-Code Database Tools

Low-code and no-code database interfaces represent different points on a spectrum of tools designed to reduce the complexity of database operations. No-code solutions eliminate coding requirements entirely, typically offering intuitive visual interfaces where users can design databases, create queries, and build applications through drag-and-drop functionality and pre-built components. These platforms are ideal for business users who lack programming expertise but need to create functional database applications rapidly.

Low-code interfaces, meanwhile, strike a balance between visual development and traditional coding. They provide graphical tools for common database tasks while still allowing developers to insert custom code when necessary for more complex operations. This hybrid approach enables technical professionals to dramatically accelerate their workflow while maintaining the flexibility to address unique requirements that purely visual tools might not accommodate.

Key Benefits for Organizations

The adoption of low-code/no-code database interfaces offers numerous advantages for organizations of all sizes. Development speed increases substantially, with projects that might have taken months now potentially completed in days or weeks. This acceleration is particularly valuable in rapidly evolving business environments where the ability to quickly adapt data systems provides a competitive edge. The democratization effect is equally significant, as these platforms enable domain experts without technical backgrounds to create and modify database applications that address their specific needs, without depending entirely on IT departments.

From a resource perspective, low-code/no-code solutions reduce the technical debt that often accumulates in traditional development environments. By generating standardized, maintainable code automatically, these platforms help organizations avoid the pitfalls of inconsistent coding practices. Additionally, they allow organizations to allocate their specialized database developers to more complex, high-value projects while empowering other team members to handle routine database tasks independently.

Navicat: Pioneering Low-Code Database Management

Among the leading solutions in this space, Navicat stands out for its comprehensive approach to low-code database management across multiple database platforms. Navicat offers several powerful features for database developers and administrators alike that significantly reduce the need to write SQL manually. For instance, its Visual Query Builder transforms the query creation process from a coding exercise to an intuitive visual experience, allowing users to construct complex SQL statements by simply dragging tables, defining joins, and selecting conditions through a graphical interface.

Navicat's data modeling tools further exemplify the low-code philosophy, enabling users to design database schemas visually and automatically generate the corresponding SQL for table creation and relationships. For data migration and synchronization tasks that would typically require extensive scripting, Navicat provides streamlined wizards that guide users through the process with minimal coding requirements. Additionally, its Stored Procedure Builder allows for the visual creation of database procedures and functions, abstracting away much of the complexity inherent in procedural SQL.

For routine database administration, Navicat reduces SQL writing through its comprehensive management interface, where common tasks like user privilege management, index creation, and performance monitoring can be accomplished through intuitive dialogues rather than manual commands. This low-code approach not only increases productivity but also reduces the likelihood of syntax errors that often occur when writing complex SQL statements manually.

The Future Landscape

As artificial intelligence and machine learning capabilities continue to evolve, low-code/no-code database interfaces are poised to become even more powerful. The integration of intelligent assistants that can suggest optimizations, predict user intentions, and even generate complex queries based on natural language descriptions represents the next frontier in this space. Furthermore, these platforms will likely incorporate more sophisticated data analysis capabilities, enabling users not just to manage data but to derive actionable insights through visual analytics tools.

Low-code/no-code database interfaces are revolutionizing how organizations interact with their data assets by removing technical barriers and accelerating development cycles. Solutions like Navicat exemplify how these platforms can empower users across technical skill levels to accomplish sophisticated database tasks efficiently. As these technologies continue to mature, they will play an increasingly pivotal role in helping organizations leverage their data for competitive advantage in an increasingly data-driven business landscape.

Today, organizations face unprecedented challenges in managing vast amounts of information from diverse sources. Traditional data modeling approaches often struggle to keep pace with the volume, variety, and velocity of modern data requirements. Data Vault 2.0 is a modern data modeling methodology specifically designed to address these challenges, offering a flexible, scalable, and auditable approach to enterprise data modeling. This article explores the core principles, components, and benefits of Data Vault 2.0, highlighting why it has become increasingly popular for large-scale data warehousing projects.

Origins and Evolution

Data Vault methodology was originally developed by Dan Linstedt in the early 2000s as a response to the limitations of traditional approaches like Kimball's dimensional modeling and Inmon's normalized models. Data Vault 1.0 introduced the core concepts of hubs, links, and satellites, creating a framework that separated business keys, relationships, and descriptive attributes. Data Vault 2.0, introduced around 2013, represents a significant evolution of the original methodology, incorporating best practices for big data, cloud computing, and agile development processes. It expanded beyond just a data modeling technique to become a comprehensive system for enterprise data warehousing.

Core Components of Data Vault 2.0

The Data Vault 2.0 architecture consists of three fundamental building blocks that form the backbone of its modeling approach:

• Hubs represent business keys and core business concepts, serving as stable anchors in the model. They contain minimal information - primarily business keys and their metadata.
• Links capture relationships between business keys, representing associations between different business entities. They are essentially many-to-many relationship tables that connect two or more hubs.
• Satellites store descriptive attributes and context about hubs or links, including historical changes. They contain time-stamped descriptive information, enabling the tracking of how data evolves over time.

This three-component structure creates a highly flexible model that can adapt to changing business requirements without requiring significant restructuring. By separating business keys from relationships and descriptive information, Data Vault 2.0 achieves a level of modularity that facilitates parallel development and integration of new data sources.

Key Principles and Benefits

Data Vault 2.0 is guided by several core principles that distinguish it from other data modeling methodologies. The approach is designed around auditability, tracking all data from source to target with complete lineage. It emphasizes scalability through its modular design, allowing organizations to expand their data warehouse incrementally without disrupting existing structures. The methodology supports adaptability to changing business requirements, a crucial advantage in today's dynamic business environment.

Organizations implementing Data Vault 2.0 often report significant benefits. The methodology enables faster integration of new data sources, sometimes reducing implementation time by 30-40% compared to traditional approaches. It provides enhanced traceability and compliance capabilities, which are increasingly important in regulated industries. Perhaps most importantly, Data Vault 2.0 creates resilient data structures that can evolve alongside the business, protecting the substantial investment that organizations make in their data infrastructure.

Implementation Considerations

While Data Vault 2.0 offers compelling advantages, implementing it requires careful planning and consideration. Organizations typically need to invest in appropriate tools and training to successfully adopt the methodology. The approach works best when implemented with automation tools that can generate and maintain the model structures, as the number of tables can grow significantly compared to other methodologies. Teams often benefit from specialized expertise, particularly during the initial phases of implementation.

Navicat Data Modeler and Data Vault 2.0

Navicat Data Modeler stands out as a powerful tool for organizations implementing Data Vault 2.0. It's ideal for designing complex data systems for various applications using Relational, Dimensional, and Data Vault 2.0 methodologies, ranging from transactional systems and operational databases to analytical platforms and data warehousing solutions. You can also use Navicat Data Modeler to effectively visualize data structures and relationships, making it easier to identify optimization opportunities and ensure alignment with business objectives.

Conclusion

Data Vault 2.0 represents a sophisticated approach to enterprise data modeling that addresses many of the limitations of traditional methodologies. By providing a flexible, scalable, and auditable framework, it enables organizations to create data warehouses that can adapt to changing business needs while maintaining historical accuracy and data lineage. As data continues to grow in both volume and strategic importance, methodologies like Data Vault 2.0 will play an increasingly crucial role in helping organizations derive maximum value from their information assets.

In recent years, traditional database systems have been struggling to keep pace with the demands of real-time analytics, IoT applications, and instantaneous decision-making, due to the increasingly complex and fast-moving data environments of modern organizations. Designed around batch processing and static data models, RDBMSes were simply not designed to handle data processing in real-time. Streaming-first architectures represent a fundamental shift in how data is captured, processed, and utilized, prioritizing continuous data flow and immediate insights over historical, retrospective analysis. This article details the rise of streaming-first architectures, examining how these innovative approaches are reshaping data processing by enabling real-time insights, continuous event streaming, and immediate actionable intelligence across diverse industries.

From Batch Processing to Streaming

The shift towards streaming-first architectures is rooted in the limitations of traditional database approaches. Historically, organizations relied on batch processing, where data would be collected, stored, and then analyzed during specific intervals. This method worked well when business cycles were slower and data volumes were more manageable. However, the digital transformation has created an environment where data is generated continuously, from millions of sources including social media, IoT devices, financial transactions, and real-time monitoring systems. Streaming-first architectures address this challenge by treating data as a continuous flow of events, allowing for immediate processing and analysis as information is generated.

Pioneering Streaming Platforms

Apache Kafka has emerged as the front-runner in streaming-first architectures, revolutionizing how organizations approach data integration and real-time processing. Originally developed by LinkedIn, Kafka provides a distributed streaming platform that can handle massive volumes of data with exceptional reliability and scalability. Companies like Uber, Netflix, and Airbnb have built entire data infrastructures around Kafka's event streaming capabilities. Apache Flink offers another powerful solution, providing sophisticated stream processing with strong consistency guarantees. These platforms enable organizations to build complex event-driven systems that can react to data in real-time, transforming how businesses make decisions and respond to changing conditions.

Traditional Databases Embrace Streaming

Recognizing the importance of streaming capabilities, many traditional database systems have begun integrating native support for streaming architectures:

• PostgreSQL, for instance, has developed extensions like pg_stream that enable real-time data ingestion and processing.
• MongoDB introduced change streams, allowing applications to access real-time data changes without the complexity of tailing the oplog.
• Oracle Database provides Oracle Stream Analytics, which enables complex event processing and real-time insights.
• Microsoft SQL Server has developed its own streaming capabilities through Azure Stream Analytics, allowing seamless integration of streaming data with traditional database operations.

Industry-Specific Applications

The impact of streaming-first architectures extends across multiple industries:

• Financial institutions use these technologies for real-time fraud detection and algorithmic trading. Manufacturing companies leverage streaming data for predictive maintenance and quality control.
• Healthcare providers implement streaming architectures to monitor patient data and enable immediate interventions.
• E-commerce platforms use streaming technologies to personalize user experiences and manage inventory in real-time.

The ability to process and act on data instantaneously has transformed these industries, creating competitive advantages for organizations that can effectively implement streaming-first approaches.

Management and Monitoring Challenges

For organizations working with these complex streaming databases and platforms, management tools have become increasingly important. For instance, Navicat provides support for managing various streaming databases, offering interfaces that can connect to and monitor different streaming platforms. This allows database administrators to oversee and optimize their streaming architectures, providing crucial visibility into data flows and system performance across different technologies and environments.

Conclusion

Streaming-first architectures represent more than just a technological trend - they signify a fundamental shift in how organizations conceptualize and utilize data. As the volume and velocity of data continue to increase, these architectures will become increasingly critical for businesses seeking to maintain competitive advantages. The ability to process and act on data in real-time is no longer a luxury but a necessity in our rapidly evolving data-driven world.

We are excited to announce that Navicat is joining the PostgreSQL Conference Germany 2025 as a Silver sponsor! As part of our ongoing commitment to the PostgreSQL community, we are proud to support this premier event and help foster innovation and collaboration among database professionals.

Event Details:

• Event: PostgreSQL Conference Germany 2025
• Date: May 89, 2025
• Venue: Berlin Marriott Hotel, Berlin, Germany

As a Silver sponsor, Navicat has been given two complimentary attendee vouchers for the conference. We want to share this opportunity with our community-so we are giving away two free tickets! If you are interested in attending PostgreSQL Conference Germany 2025, simply contact us at media@navicat.com for your chance to receive a ticket. Tickets will be distributed on a first-come, first-served basis.

Navicat is a strong supporter of the PostgreSQL community and is committed to backing more PostgreSQL events around the world. Stay tuned to our blog for updates on future events and more chances to win free tickets!

Database-as-a-Service (DBaaS) has been a cornerstone of cloud computing for over a decade, but recent developments have significantly expanded its capabilities and reach. While the core concept of delivering managed database services in the cloud is not new, the past few years have witnessed remarkable innovations that are reshaping how organizations approach data management. This article explores several noteworthy advancements in the DBaaS landscape, from the emergence of truly serverless database offerings to the integration of artificial intelligence for autonomous operations. We'll examine how these developments are transforming the economics of database management, enabling new use cases, and providing organizations with unprecedented flexibility in how they deploy and manage their data infrastructure across multiple environments.

The Rise of Serverless Databases

Perhaps the most transformative recent trend in DBaaS is the rise of truly serverless database offerings. Unlike earlier cloud database models that required some level of capacity planning, serverless databases automatically scale compute and storage resources in response to workload demands - all the way down to zero during periods of inactivity. AWS Aurora Serverless, Azure SQL Database serverless, and MongoDB Atlas Serverless have pioneered this approach, introducing consumption-based pricing models that align costs directly with actual usage. This model eliminates the need for capacity planning and removes the overhead of managing database resources, allowing development teams to focus entirely on application logic rather than infrastructure concerns.

AI-Powered Database Management

The integration of artificial intelligence and machine learning capabilities directly into database services represents another frontier in DBaaS evolution. Cloud providers now offer databases with built-in intelligence for query optimization, anomaly detection, and predictive scaling. Oracle Autonomous Database, for instance, uses machine learning to automate routine administration tasks like tuning, security patching, and backup, while Microsoft's Azure SQL Database employs AI to detect potential performance issues before they impact applications. These intelligent capabilities effectively transform databases from passive data repositories into active systems that continuously optimize themselves without human intervention.

Multi-Cloud and Hybrid Deployments

Multi-cloud and hybrid cloud database solutions have emerged as a response to growing concerns about vendor lock-in and the need for deployment flexibility. Services like CockroachDB, MongoDB Atlas, and DataStax Astra now provide consistent database experiences across multiple cloud environments and on-premises infrastructure. This approach gives organizations the freedom to deploy databases wherever makes the most business sense while maintaining operational consistency. For global enterprises with diverse regulatory requirements or legacy infrastructures, these multi-cloud databases offer a path to cloud adoption that doesn't compromise on deployment flexibility or data ownership concerns.

Specialized Database Services

The specialized database revolution continues to accelerate in the DBaaS space, with purpose-built database services optimized for specific data models and workloads. Time series databases like InfluxDB Cloud and TimescaleDB address the unique requirements of temporal data. Graph databases such as Neo4j Aura and Amazon Neptune provide native support for relationship-centric data models. Vector databases including Pinecone and Weaviate deliver high-performance similarity search for AI applications. This specialization trend acknowledges that different data workloads have distinct requirements that general-purpose databases struggle to address efficiently, leading to a variety of purpose-built services tailored to specific use cases.

Unified Database Management Tools

For organizations working with these diverse cloud database services, management tools like Navicat have evolved to provide unified interfaces for working with multiple database platforms across different cloud environments. Navicat supports connections to various cloud databases including Amazon RDS, Azure SQL Database, and Google Cloud SQL, allowing database administrators to seamlessly manage their cloud databases alongside on-premises systems. This centralized approach to database management significantly streamlines operations for teams working with heterogeneous database environments, providing consistent tools for schema design, query execution, and performance monitoring across the increasing varieties of cloud database services.

Conclusion

As we look toward the future of DBaaS, the line between different database models will likely continue to blur as services incorporate multiple data models within unified platforms. The emphasis on operational simplicity, automatic optimization, and consumption-based models will only strengthen as cloud providers compete to deliver optimal data management experiences. For organizations embarking on digital transformation initiatives, these advancements in DBaaS technology offer unprecedented opportunities to harness the power of data without the traditional burdens of database administration.

Time-Series Databases (TSDBs) have emerged as a specialized solution to one of modern computing's most significant challenges: the efficient storage, retrieval, and analysis of time-based data. As organizations' collection of data from sensors, applications, and systems that generate readings at regular intervals have increased, the limitations of traditional database systems for handling this type of data have become apparent.

Traditional relational database management systems (RDBMS) were designed for transactional workloads where relationships between different entities matter more than the temporal aspect of the data. While these systems can certainly store time-stamped data, they aren't optimized for the high-frequency writes, temporal queries, and data lifecycle management associated with time-series workloads. This limitation created the need for purpose-built solutions that could handle the unique characteristics of time-series data. This article explores how traditional and time-series database technologies integrate and complement each other, examining various implementation approaches.

The Integration of Traditional and Time-Series Databases

The evolution of TSDBs hasn't occurred in isolation from traditional database technologies. Rather, there has been a gradual integration of time-series capabilities into existing database frameworks, as well as the development of standalone systems that borrow concepts from traditional databases. This symbiotic relationship has led to a spectrum of solutions ranging from pure-play TSDBs to traditional databases with time-series extensions.

One of the most notable examples of this integration is TimescaleDB, which extends PostgreSQL to handle time-series data efficiently. By building on PostgreSQL's solid foundation, TimescaleDB inherits the reliability, SQL compatibility, and rich ecosystem of a mature RDBMS while adding specialized time-based indexing, automated partitioning, and optimized compression algorithms. This hybrid approach allows organizations to maintain a single database system for both relational and time-series data, reducing operational complexity.

Similarly, major database vendors like Microsoft and Oracle have incorporated time-series capabilities directly into their flagship products. Microsoft SQL Server offers temporal tables that track the history of data changes over time, while Oracle Database includes features specifically designed for managing time-series data within the context of a traditional RDBMS.

Complementary Approaches and Cloud Solutions

Beyond extensions to existing systems, many organizations adopt a complementary approach where traditional databases and dedicated TSDBs coexist within their data architecture. In these scenarios, operational data might reside in a traditional RDBMS like MySQL or Oracle, while high-frequency metrics, logs, and other time-stamped data are routed to specialized TSDBs like InfluxDB, Prometheus, or Graphite. Integration layers, often implemented through ETL (Extract, Transform, Load) processes or API-based data exchange, ensure that information can flow between these systems when cross-domain queries are required.

The rise of cloud computing has further blurred the lines between traditional and time-series databases. Managed services like Amazon Timestream, Azure Data Explorer, and Google Cloud's BigQuery are designed to handle time-series workloads at scale while maintaining compatibility with traditional SQL-based query languages. These services abstract much of the underlying complexity, allowing developers to work with time-series data by leveraging familiar concepts from traditional database systems.

Managing Diverse Database Ecosystems with Navicat

For database administrators and developers tasked with managing this increasingly diverse systems, tools like Navicat provide a unified interface for interacting with multiple databases. Navicat's versatility allows it to connect to both traditional RDBMS platforms like MySQL, PostgreSQL, and SQL Server, as well as newer time-series focused systems that offer SQL-compatible interfaces. Through Navicat, administrators can visually design schemas, write and test queries, and monitor performance across their entire database network.

Conclusion

The relationship between traditional databases and time-series databases is not one of replacement but of evolution and integration. Organizations today have multiple options for handling time-series data, from specialized standalone solutions to extensions of familiar database systems. As data volumes continue to grow and real-time analytics become increasingly important, we can expect further innovation in how these systems interact and complement each other. The ability to effectively manage these diverse database technologies through tools like Navicat will remain crucial for organizations seeking to derive maximum value from their time-based data.

As organizations face increasing pressure to protect sensitive data while making it accessible to those who need it, database systems have evolved to incorporate sophisticated privacy-preserving features. These advancements represent a fundamental shift in how we approach data security, moving beyond simple encryption to provide comprehensive protection throughout the data lifecycle. This article explores how modern databases implement privacy protection and examines the practical implications for organizations managing sensitive information.

The Evolution of Database Privacy

Database privacy has evolved significantly from the early days of basic access controls. Modern privacy-preserving features operate at multiple levels, from the individual field to entire databases, providing granular control over how sensitive information is stored, accessed, and used. This layered approach ensures that organizations can maintain data utility while meeting stringent privacy requirements and regulatory standards.

PostgreSQL's Advanced Privacy Features

PostgreSQL leads the way in open-source database privacy with its comprehensive suite of security features. Its row-level security policies allow organizations to implement fine-grained access controls based on user context, ensuring that individuals can only access the specific data rows they're authorized to see. Column-level encryption adds another layer of protection by securing sensitive fields while keeping others accessible for analysis. These features enable organizations to implement sophisticated privacy policies without sacrificing database functionality.

Oracle's Enterprise Privacy Solutions

Oracle's database system brings enterprise-grade privacy features to large organizations. Its Transparent Data Encryption protects data at rest without requiring application changes, while Data Redaction enables dynamic masking of sensitive information based on user context. The Oracle Database Vault provides additional protection by restricting privileged user access to application data, preventing even database administrators from viewing sensitive information when unnecessary.

Microsoft SQL Server's Privacy Innovation

Microsoft SQL Server has introduced groundbreaking privacy features with its Always Encrypted capability, which enables clients to encrypt sensitive data inside application programs without revealing encryption keys to the database engine. This approach, combined with Dynamic Data Masking and Row-Level Security, provides a comprehensive privacy framework that organizations can implement across their data infrastructure.

MongoDB's Modern Approach

MongoDB has embraced modern privacy requirements with its Client-Side Field Level Encryption, allowing organizations to encrypt sensitive data before it even reaches the database server. Its Queryable Encryption feature takes this a step further by enabling queries on encrypted data without decryption, representing a significant advancement in privacy-preserving database technology.

Implementation and Management

Implementing privacy-preserving features requires careful planning and ongoing management. Organizations must balance privacy requirements with performance considerations, ensuring that protection mechanisms don't unduly impact database operations. Tools like Navicat play a crucial role in this process, providing unified interfaces for managing privacy features across different database platforms and helping organizations maintain consistent privacy policies throughout their data infrastructure. This centralized management approach is particularly valuable for organizations running multiple database systems, as it provides consistent tools and workflows for privacy management across their entire database infrastructure. Navicat's support for secure connections and encrypted sessions adds another layer of protection, ensuring that database management activities themselves don't create security vulnerabilities.

The Future of Database Privacy

As privacy concerns continue to grow and regulations become more stringent, we can expect database systems to develop even more sophisticated privacy-preserving features. The trend toward integrated privacy protection, where security is built into the core database architecture rather than added as an afterthought, will likely accelerate. Organizations that embrace these advanced features, while maintaining effective management through tools like Navicat, will be well-positioned to protect sensitive data from competitors as well as from malicious third parties.

In the world of data management, organizations have long struggled with the complexity and time-consuming nature of Extract, Transform, and Load (ETL) processes. Zero-ETL databases have emerged as a revolutionary solution to this challenge, promising to eliminate the traditional barriers between operational and analytical data systems. In this article, we'll learn how Zero-ETL databases work as well as examine the evolving role of traditional databases in modern data processing.

Understanding Zero-ETL Databases

Zero-ETL databases represent a fundamental shift in how we think about data integration. Instead of explicitly moving and transforming data between systems, these databases create direct pathways for data access and analysis. Think of it as replacing a manual assembly line with an automated production system - the end result is the same, but the process becomes seamless and immediate.

Major cloud providers have begun implementing Zero-ETL capabilities in their offerings. Snowflake provides native application integration, allowing direct data access without traditional ETL processes. Google BigQuery offers streamlined data integration capabilities, while Amazon Redshift has developed Zero-ETL integration with their Aurora database service. These solutions aim to make real-time analytics possible without the overhead of data movement.

The Role of Traditional Databases

Traditional databases continue to play an essential part in Zero-ETL architectures, often serving as primary data sources.

• PostgreSQL, which offers enterprise-grade reliability and sophisticated data handling capabilities, frequently acts as a source database for Zero-ETL systems. Its advanced features enable direct integration with platforms like Snowflake and Amazon Redshift, allowing analytical queries without traditional data movement.
• MySQL participates in Zero-ETL scenarios through native connectors and real-time change data capture systems. For example, Amazon's Aurora MySQL can share data with Redshift without explicit ETL processes, enabling immediate analysis of operational data. This integration preserves the strengths of MySQL while extending its analytical capabilities.
• MongoDB brings its document-oriented approach to Zero-ETL architectures through features like Atlas Data Federation and change streams. These capabilities allow applications to access and analyze data directly from MongoDB without extracting it to separate analytical systems.
• Redis, while primarily known as a high-performance cache, serves a unique role in Zero-ETL architectures: it acts as an intermediate layer that accelerates data access without requiring explicit ETL processes.

Benefits and Considerations

The transition to Zero-ETL approaches offers significant advantages. Organizations can analyze data in real-time without waiting for ETL jobs to complete. This immediacy supports faster decision-making and more responsive business operations. The elimination of explicit ETL processes also reduces the potential for errors and decreases the maintenance burden on data teams.

However, implementing Zero-ETL solutions requires careful planning. Organizations must consider data consistency requirements, query performance expectations, and the specific capabilities of their chosen platforms. The role of traditional databases becomes even more critical in this context, as they must support both operational requirements and real-time analytical access.

Organizations using Navicat can manage their local and cloud database instances alongside Zero-ETL databases, creating a unified management experience across their data infrastructure.

Looking Forward

As Zero-ETL databases continue to evolve, we can expect to see even tighter integration with traditional database systems. We are also likely to see the boundaries between operational and analytical data blur at an accelerated rate. Organizations that embrace these technologies, while maintaining their expertise with traditional databases through tools like Navicat, will be well-positioned to handle tomorrow's data challenges.

In today's data-driven business landscape, organizations face the challenge of managing both day-to-day transactions and complex analytics within their database systems. Traditionally, these workloads were handled separately: Online Transaction Processing (OLTP) systems managed operational data, while Online Analytical Processing (OLAP) systems handled reporting and analysis. Hybrid Transactional/Analytical Processing (HTAP) has been gaining traction as a revolutionary approach that combines these capabilities into a unified system, enabling real-time analytics on operational data without the complexity and delays of traditional data warehousing. This blog article explores the fundamentals of HTAP architecture, examines how traditional databases have evolved to support HTAP capabilities, and discusses the role of database management tools in implementing HTAP solutions.

Fundamentals of HTAP Architecture

The fundamental principle behind HTAP is straightforward: maintain a single source of truth that can efficiently handle both transactional and analytical workloads. This approach eliminates the need for Extract, Transform, Load (ETL) processes and reduces data latency, enabling organizations to make decisions based on the most current information available. HTAP systems achieve this through sophisticated architecture that typically includes in-memory processing, columnar storage capabilities, and advanced workload management mechanisms.

Traditional Databases and HTAP

While purpose-built HTAP databases like SAP HANA and MemSQL lead the market, traditional databases have evolved to support HTAP workloads in various capacities. MongoDB, for instance, has embraced HTAP through its aggregation pipeline and change streams features. These capabilities allow organizations to perform real-time analytics on operational data while maintaining MongoDB's core strengths in handling document-based transactions. The platform's ability to scale horizontally makes it particularly suitable for organizations dealing with large volumes of semi-structured data.

PostgreSQL, often praised for its extensibility, offers several paths to HTAP functionality. Through its Foreign Data Wrapper (FDW) feature, PostgreSQL can integrate with specialized analytical stores while maintaining transactional capabilities. The TimescaleDB extension transforms PostgreSQL into a powerful time-series database, enabling complex analytical queries without sacrificing transactional performance. Additionally, the Citus extension provides distributed query capabilities, allowing PostgreSQL to scale both transactional and analytical workloads across multiple nodes.

MySQL, particularly through its NDB Cluster technology, is well suited to HTAP. The system maintains separate nodes for transactions and analytics, with real-time replication ensuring data consistency. The InnoDB storage engine's buffer pool optimizations and support for in-memory tables further enhance analytical performance without compromising transactional integrity. MySQL's Group Replication feature allows organizations to dedicate specific nodes to analytical workloads, providing a flexible approach to HTAP implementation.

Database Management Tools for HTAP

For organizations implementing HTAP solutions using these traditional databases, tools like Navicat prove invaluable for database management and monitoring. Navicat's unified interface supports multiple database systems, making it easier to manage hybrid environments where different databases might be employed for various aspects of the HTAP architecture. Its visual query builder and data modeling tools help developers and database administrators optimize both transactional and analytical workloads.

Conclusion

The future of HTAP looks promising as traditional database systems continue to evolve and incorporate more sophisticated HTAP capabilities. The growing demand for real-time analytics, coupled with advancements in hardware and software technologies, is driving innovation in this space. Organizations are increasingly recognizing that the ability to perform real-time analytics on operational data is not just a competitive advantage but a necessity in today's fast-paced business environment.

As we move forward, the distinction between transactional and analytical systems may continue to blur, with HTAP becoming the standard approach for database architecture. This evolution will likely be accompanied by further improvements in traditional databases' HTAP capabilities, making sophisticated real-time analytics more accessible to organizations of all sizes.

Back in August of 2024, Navicat released version 17.1, which added Enhanced Query Explain and Expanded Database Connectivity. Now, version 17.2 is in Beta and is slated for release shortly. Some of the new features that we'll be talking about in today's blog include:

• AI Assistant:
  Get AI assistance directly within Navicat whenever you need it. Enable you to ask questions and receive instant answers.
• Integrated Snowflake support:
  Navicat has now added support for Snowflake, thereby enhancing the management of your cloud-based data warehousing platform.
• Data Vault and Dimensional modeling methods:
  Build scalable and adaptable data architectures for various applications using Relational, Dimensional, and Data Vault methodologies.
• Data profiling in BI workspace data source:
  There's an integrated data profiling tool in Data Preview to provide a visual and comprehensive view of your data.

The rest of the blog will explore each of these in greater detail.

AI Assistant

An AI Assistant is an integrated tool that provides instant, contextual guidance and answers within a software application, leveraging artificial intelligence to help users solve problems, understand features, and improve their workflow through natural language interactions. Navicat's AI Assistant helps you write your SQL statements more efficiently. It does this by submitting your inquiries to the AI providers for processing, with responses sent exclusively back to the Navicat application installed on your local device. You can receive guidance from many of the popular AI chatbots, including ChatGPT, Google Gemini, DeepSeek, and Ollama.

Once enabled, you can access the AI Assistant from the right-hand AI pane:

Integrated Snowflake Support

Snowflake is a cloud-native data warehousing platform that allows organizations to store, process, and analyze large volumes of structured and semi-structured data. It separates storage and compute resources, enabling flexible, scalable data management across multiple cloud platforms like AWS, Azure, and Google Cloud, with built-in performance optimization and easy data sharing capabilities.

Now you can manage your Snowflake tables, views, queries and functions directly from Navicat.

Data Vault and Dimensional Modeling Methods

Data Vault and Dimensional modeling are two strategic approaches to designing data models for business intelligence. Data Vault focuses on historical tracking with flexible components like Hubs, Links, and Satellites, allowing adaptable management of complex data environments. Dimensional modeling, in contrast, structures data into fact and dimension tables, optimizing for query performance and user comprehension by providing clear, contextual insights into quantitative business data.

You'll find the new model types on the New Model dialog:

Data Profiling in BI Workspace Data Source

Introduced in Navicat 17, the Data Profiling tool provides a visual and comprehensive view of your data at the click of a button! Now the BI workspace's Data Preview includes the same Data Profiling abilities.

Other Improvements

In addition to the new features outlined above, Navicat 17.2 also includes a few notable improvements. These include:

• Filter & Sort pane in data viewer now supports text mode
• Structure Synchronization includes MongoDB databases
• Several other usability enhancements

Conclusion

This blog covered just some of the exciting new features to expect in Navicat 17.2. Be sure to check the Navicat homepage for updates on the release!

The realm of data storage has evolved dramatically over the past decade, leading organizations to seek more effective ways to manage their data assets. Database Lakehouse Architecture has emerged as an innovative solution that bridges the gap between traditional data warehouses and data lakes, combining the best aspects of both approaches. This article explores how Lakehouse Architecture works and examines the crucial role that traditional databases play in supporting these modern data platforms.

Lakehouse Architecture Defined

A Lakehouse Architecture represents a new approach to data management that merges the flexibility and cost-effectiveness of data lakes with the reliability and performance of data warehouses. At its core, a Lakehouse uses cloud object storage to maintain vast amounts of raw data in open file formats like Apache Parquet, while implementing additional layers of functionality to provide warehouse-like features such as ACID transactions, schema enforcement, and optimized query performance.

The Foundation: Storage and Processing

The foundation of a Lakehouse typically consists of cloud object storage systems that house data in open formats. These systems are enhanced by table formats like Delta Lake, Apache Hudi, or Apache Iceberg, which add crucial capabilities for managing data reliability and consistency. This combination creates a robust base layer that can handle both structured and unstructured data while maintaining the performance characteristics needed for enterprise applications.

Query Engines and Processing Layer

Above the storage layer, powerful query engines like Apache Spark and Trino provide the computational muscle needed to process and analyze data efficiently. These engines can handle everything from basic SQL queries to complex machine learning workloads, making the Lakehouse suitable for a wide range of analytical needs. Managed solutions like Databricks SQL and Snowflake further enhance these capabilities by providing optimized, enterprise-grade query processing.

The Role of Traditional Databases

While the core Lakehouse infrastructure handles large-scale data storage and processing, traditional databases play crucial supporting roles in the overall architecture. PostgreSQL, with its ACID compliance and rich feature set, often serves as the operational database for structured data that requires frequent updates and complex transactions. Its ability to handle both relational and JSON data makes it particularly valuable in modern data architectures.

MongoDB comes into play when applications need to handle semi-structured data with flexible schemas. Its document-oriented approach complements the Lakehouse by providing a repository for application-specific data storage. This makes it particularly valuable for microservices architectures that feed data into the Lakehouse.

Redis serves as a high-performance caching layer, dramatically improving data access speeds for frequently accessed information. Its in-memory architecture and support for diverse data structures make it ideal for maintaining real-time views of data that originates from the Lakehouse, enabling fast application responses while maintaining consistency within the broader ecosystem.

Management and Integration

Managing the complex Lakehouse infrastructure requires sophisticated tools, and this is where database management tools like Navicat prove invaluable. Navicat provides comprehensive support for the traditional databases involved within Lakehouse architectures, offering unified interfaces for managing PostgreSQL, MongoDB, Redis, and other databases that play crucial roles in the overall system. This integration capability helps organizations maintain consistency and efficiency across the entire data infrastructure.

Future Outlook

The Lakehouse Architecture continues to evolve, with new tools and capabilities emerging regularly. The integration of traditional databases with modern Lakehouse platforms represents a pragmatic approach to enterprise data management, combining the strengths of established database systems with the innovation of modern data platforms. As organizations continue to deal with growing data volumes and increasingly complex analytical requirements, Lakehouse Architecture, supported by traditional databases and modern management tools like Navicat, provides a solid foundation for future data management needs.

In today's microservices-driven world, organizations face increasing challenges in managing data across distributed systems. Database Mesh Architecture has emerged as a powerful solution to these challenges, offering a decentralized approach to data management that aligns with modern application architectures. This article explores how Database Mesh Architecture works and how to implement it using popular databases such as PostgreSQL and MongoDB.

What Exactly Is Database Mesh Architecture?

Database Mesh Architecture represents a decentralized approach to managing data infrastructure where different databases work together as a cohesive system while remaining independently operated. Unlike traditional monolithic database systems, a database mesh distributes data management across multiple specialized databases, each serving specific business domains or use cases. This approach enables organizations to maintain flexibility while ensuring data consistency and accessibility across the entire system.

Core Principles and Components

At its heart, Database Mesh Architecture operates on the principle of domain-oriented data ownership. Each business domain maintains control over its data and database choices, enabling teams to make independent decisions about data structures and management approaches. This autonomy is balanced with standardized practices that ensure system-wide coherence.

The architecture also emphasizes self-service infrastructure, where database resources can be provisioned automatically according to predefined standards. This automation reduces operational overhead while maintaining consistent security and performance standards across the mesh.

An essential component is the interoperability layer, which enables seamless communication between different database systems. This layer handles standardized data access protocols, implements consistent security policies, and manages metadata across the entire mesh. Through this layer, different database systems can work together effectively while maintaining their specialized roles.

Implementing a Database Mesh with Popular Databases

A successful database mesh implementation combines various database types to serve different needs:

• PostgreSQL often serves as the foundation for transactional data, offering strong ACID compliance, sophisticated partitioning capabilities, and advanced replication features. Moreover, its many extensions makes it particularly valuable in a mesh architecture, where flexibility and extensibility are crucial.
• For document-oriented data, MongoDB provides excellent capabilities with its flexible schema design and horizontal scaling features. Its native support for JSON documents and built-in sharding capabilities make it ideal for handling varied and evolving data structures within the mesh.
• High-performance caching requirements are typically addressed using Redis, which excels at in-memory data storage and real-time operations. Its pub/sub capabilities and cluster mode for scaling make it an excellent choice for managing fast-changing data within the mesh.
• Search functionality is often implemented using Elasticsearch, which provides powerful full-text search capabilities along with analytics features. Its distributed architecture naturally aligns with the mesh concept, enabling efficient data processing across the system.

Tips For Implementation and Management

When implementing a database mesh, organizations should start with a modest scope, focusing on a few well-defined domains before expanding. This approach allows teams to validate patterns and practices before scaling the architecture. Standardization plays a crucial role in successful implementation, particularly in areas of naming conventions, security practices, and data ownership concerns.

Continuous monitoring and optimization are essential for maintaining mesh performance. Teams should track key metrics, monitor data consistency, and regularly optimize based on observed usage patterns. This ongoing attention ensures the mesh remains efficient and effective as business needs evolve.

Unsurprisingly, the complexity of a database mesh requires sophisticated management tools. Navicat stands out by providing comprehensive support for most databases commonly used in mesh architectures. Through its interface, teams can perform visual database design, query optimization, data synchronization, and performance monitoring across different database systems. This unified management approach greatly simplifies the operation of complex mesh architectures.

Conclusion

Database Mesh Architecture represents a sophisticated approach to handling complex data requirements in distributed systems. By thoughtfully combining different database technologies and managing them with professional-grade tools like Navicat, organizations can build flexible, scalable data infrastructures that meet modern business needs while maintaining manageability and performance.

The landscape of data storage and management is currently undergoing a dramatic transformation. As organizations deal with increasingly diverse types of data, traditional relational databases are no longer sufficient for many modern applications. Enter multi-modal databases, a powerful solution that's reshaping how we think about data storage and manipulation. This article explores how multi-modal databases are revolutionizing data management by enabling organizations to store and process multiple types of data - from traditional tables to documents, graphs, and vectors - all within a single, unified system.

What Are Multi-Modal Databases?

A multi-modal database is a database management system designed to handle multiple types of data models within a single, integrated backend. Unlike traditional relational databases that primarily work with structured data in tables, multi-modal databases can simultaneously manage different data types and structures - from documents and graphs to vectors and spatial data.

For instance, consider an e-commerce platform. It might need to store product information in a traditional tabular format, customer reviews as documents, recommendation systems as vectors, and relationship networks as graphs. A multi-modal database can handle all these requirements within a single system, eliminating the need for multiple specialized databases.

The Evolution from Traditional Databases

Traditional relational databases were designed to work with structured data. As such, they excel at handling relationships between well-defined data entities through tables and SQL queries. However, traditional databases face limitations when dealing with unstructured data like documents or images, complex relationships better represented as graphs, vector embeddings for AI/ML applications, and semi-structured data with varying attributes.

Multi-modal databases address these limitations by incorporating different data models into a unified system. Modern database platforms like MongoDB and PostgreSQL have evolved to handle multiple data models effectively.

Key Features and Benefits

Multi-modal databases offer several advantages over traditional systems:

Flexibility: They can adapt to varying data requirements without needing multiple specialized databases. PostgreSQL, for example, supports traditional relational data alongside JSON documents and, more recently, vector storage for AI applications.

Simplified Architecture: Organizations can reduce complexity by using a single database system instead of maintaining multiple specialized databases. This consolidation, supported by tools like Navicat, makes database management more straightforward and efficient.

Improved Performance: By handling different data models natively, multi-modal databases can optimize performance for each type of data while maintaining data consistency across models.

Cost Efficiency: Using a single database system instead of multiple specialized ones can significantly reduce operational costs and complexity.

Real-World Applications

The versatility of multi-modal databases makes them ideal for modern applications such as:

Social Media Platforms: Storing user profiles as documents, friendship networks as graphs, and media content metadata in traditional tables.

Healthcare Systems: Managing patient records as documents, medical imagery metadata in tables, and treatment relationship networks as graphs.

AI-Powered Applications: Storing traditional data alongside vector embeddings for machine learning models, particularly in recommendation systems and natural language processing applications.

The Role of Modern Database Tools

Database management tools have evolved alongside these multi-modal systems. Navicat, for instance, provides comprehensive support for both MongoDB and PostgreSQL, offering intuitive interfaces for managing different data models within these platforms. This support includes visual query builders, data modeling tools, and automation capabilities that work across different data models.

Conclusion

In this exploration of multi-modal databases, we've seen how they fundamentally differ from traditional relational databases by supporting diverse data types within a single system, from documents and graphs to vectors and spatial data. We've examined their key benefits, including increased flexibility, simplified architecture, improved performance, and cost efficiency, while exploring real-world applications across social media, healthcare, and AI-powered systems.

As organizations continue to deal with increasingly diverse data types, multi-modal databases represent a significant evolution in data management. Their ability to handle various data models efficiently, combined with support from versatile management tools like Navicat, makes them an invaluable solution for modern data challenges. Whether you're working with traditional relational data, documents, graphs, or vectors, multi-modal databases provide a unified, efficient approach to data management.

According to the 2023 Stack Overflow Developer Survey, PostgreSQL has achieved a significant milestone by overtaking MySQL as the most admired and desired database system among developers. This shift reflects a growing appreciation for PostgreSQL's robust feature set, reliability, and extensibility in the developer community.

This changing landscape has sparked innovation in the database-as-a-service space, particularly evident in the competition between two cutting-edge platforms: PlanetScale, built on MySQL, and Neon, powered by PostgreSQL. Both services are reimagining how developers interact with databases in the cloud era. These developments should be of interest to Navicat users, as both are fully supported by Navicat's comprehensive database development and management tools.

This blog will provide a comparison of the two services and offer some tips for choosing between them.

PlanetScale: MySQL's Modern Evolution

PlanetScale brings MySQL into the serverless age, leveraging Vitess, the same technology that powers YouTube's database infrastructure. Its standout features include database branching (similar to Git workflows), non-blocking schema changes, and automated scaling capabilities. Developers particularly appreciate PlanetScale's deployment workflow, which allows them to create development branches, make schema changes, and deploy with confidence through automated review processes.

The platform excels in:

• Developer-friendly database branching
• Seamless schema management
• Proven scalability
• Built-in connection pooling
• Zero-downtime schema changes

Neon: PostgreSQL's Serverless Innovation

Neon takes PostgreSQL's rising popularity and combines it with modern cloud architecture. It separates storage from compute, enabling true serverless scaling and instant database branching. Neon maintains full compatibility with PostgreSQL while adding cloud-native features that developers expect in modern platforms.

Key advantages include:

• Full PostgreSQL feature compatibility
• Efficient storage architecture
• Serverless autoscaling
• Instant branching capabilities
• Cost-effective resource utilization

Choosing Between the Platforms

The choice between PlanetScale and Neon often aligns with specific project needs and team expertise. PlanetScale is particularly attractive for teams with MySQL experience who need proven scalability and appreciate Git-like workflows. Its schema management tools and deployment safety features make it especially suitable for teams working on rapidly evolving applications.

Meanwhile, Neon appeals to developers who prefer PostgreSQL's advanced features and want to leverage them in a serverless environment. Its storage-compute separation and efficient resource utilization make it particularly cost-effective for applications with variable workloads.

Conclusion

PlanetScale and Neon represent the future of database management, offering developers powerful tools to build and scale applications without the operational overhead of traditional database management. Their emergence highlights how the database landscape is evolving to meet modern development needs, with both MySQL and PostgreSQL finding new ways to serve developers through innovative platforms.

The competition between these platforms showcases how the database market continues to evolve, with each solution bringing unique strengths to the table. As PostgreSQL's popularity continues to grow, as evidenced by the Stack Overflow survey, we can expect to see continued innovation in both services. Moreover, the availability of professional grade database management tools like Navicat, which supports both PlanetScale and Neon, ensures that developers can maintain their preferred workflow regardless of their platform choice. Navicat's comprehensive toolset, combined with the innovative features of both platforms, provides developers with all of the necessary tools for building and managing modern applications.

Range Types

It's no secret that PostgreSQL is one of the most flexible databases on the market. In fact, PostgreSQL's extensibility and rich feature set recently propelled PostgreSQL ahead of MySQL as the most admired and desired database system among developers. In this series on creating custom data types in PostgreSQL using Navicat Premium 17 we've explored a few options so far, including custom Domains, as well as Composite and Enumerated types. The topic of this week's blog will be Range types, which are particularly useful when you need to work with continuous intervals or ranges of values.

A Quick Description of the RANGE TYPE

Range Types in PostgreSQL provide a means for working with continuous intervals of values. Hence, a range could include all product prices between $10 and $20. These ranges let you work with any values that fall within their bounds, making it easy to check for things like scheduling conflicts or price matching. Ranges are particularly useful in databases when you need to work with continuous spans of time, numerical intervals, or any other sequential data.

For example, in a movie theater's database, you might use ranges to represent screening times, ensuring no two movies are scheduled to overlap in the same theater. Or in a hotel booking system, ranges could track room availability dates, making it easy to check for vacancy conflicts. Range types are especially valuable because PostgreSQL handles all the complex logic of comparing and manipulating these intervals, providing built-in operations to check for overlaps, containment, and intersections between ranges.

In the next section, we'll go over a couple of practical examples using Navicat Premium 17 and the free DVD Rental database.

Defining Film Runtime Ranges

Before considering a custom range type, we should check is one of PostgreSQL's built-in range types would accomplish what we're looking for. These include:

• int4range: Range of integer
• int8range: Range of bigint
• numrange: Range of numeric
• tsrange: Range of timestamp without time zone
• tstzrange: Range of timestamp with time zone
• daterange: Range of date

Although film runtimes in the DVD Rental database are stored as integers, creating our own range type makes sense when we have specific business requirements that aren't covered by built-in types. For instance, if we were tracking film runtime ranges with special validation rules:

-- Creating a custom minutes range type with specific validationCREATE TYPE runtime_range AS RANGE (    subtype = integer,    subtype_diff = int4mi);CREATE TABLE film_runtime_categories (   category_name VARCHAR(50),   typical_runtime runtime_range,   CHECK (lower(typical_runtime) >l= 30 AND upper(typical_runtime) <= 240));-- Adding rows to the tableINSERT INTO film_runtime_categories VALUES    ('Short Film', '[30,45]');INSERT INTO film_runtime_categories VALUES    ('Feature Film', '[75,180]');

Creating a Range Type in Navicat 17

An easier way to define a custom Type is to use Navicat's GUI-based tools. You'll find them in both Navicat Premium 17 and Navicat for PostgreSQL 17. To access the Type tool, simply click "Others" in the main toolbar and then select "Type" from the drop-down:

That will bring up the Objects pane, where we'll see a list of existing types. To create a new one, click on the arrow next to the "New Type" item in the "Objects" toolbar and select the "Range" item from the context menu:

The Range Type designer has three tabs: General, Comment, and SQL Preview. On the General tab, the main details that we need to supply are the "Subtype" and "Subtype Diff". We'll base our type on the int4 as follows:

Before clicking the "Save" button we can take a look at the statement that Navicat will generate by clicking on the "SQL Preview" tab:

Notice that the Type name is "Untitled" since we haven't yet saved the definition. That is expected.

Upon clicking on the "Save" button, we are presented with a "Save As" dialog where we can give our Type a name. Let's call it "runtime_range":

We can now use our "runtime_range" type just like any other PostgreSQL data type. For instance, if we create the "film_runtime_categories" table that we saw in the example above, we can set the "typical_runtime" column to our custom type by selecting it from the "Object Type" drop-down(s):

We can then add our field validation on the Checks tab:

Conclusion

In today's blog, we created a Range Type using Navicat Premium 17's Type tool and created a new table that featured our custom type. In Part 5 we will conclude the series by extending the Base Type.

Enumerated Types

In this series on creating custom data types in PostgreSQL using Navicat Premium 17 we've explored a couple of options so far. In part 1, we learned how to create a custom Domain for the free DVD Rental database. Last week, we created a Composite Type to return complex data from a user-defined function. Today's blog will cover Enumerated Types, which limit values a set of predefined options.

A Quick Overview of the ENUM TYPE

Enumerated types (ENUMs) allow us to define a data type with a static, ordered set of values. This is useful for situations where a column must contain one of a limited set of predefined values.

Like other PostgreSQL types, the ENUM type is created using the CREATE TYPE statement. Here's an ENUM that defines four user statuses:

CREATE TYPE user_status AS ENUM ('active', 'inactive', 'suspended', 'pending');

Here's another that defines movie ratings:

CREATE TYPE movie_rating AS ENUM ('G', 'PG', 'PG-13', 'R', 'NC-17');

Once defined, we can use our custom type in a table as follows:

CREATE TABLE films (  film_id SERIAL PRIMARY KEY,  title VARCHAR(255),  rating movie_rating);

Creating an Enumerated Type in Navicat 17

An easier way to define an Enumerated Type is to use Navicat's GUI-based tools. You'll find them in both Navicat Premium 17 and Navicat for PostgreSQL 17. To access the Type tool, simply click "Others" in the main toolbar and then select "Type" from the drop-down:

That will bring up the Objects pane, where we'll see a list of existing types. To create a new one, click on the arrow next to the "New Type" item in the "Objects" toolbar and select the "Enum" item from the context menu:

That will launch the Type designer in a new tab. On the General tab there will be an empty cell in which we can enter the first Label for our Enum, i.e., "G":

We can add a new row to enter the next Label by clicking on "Add Label". Once all the Labels have been entered, the General tab should look like this:

Before clicking the "Save" button we can take a look at the statement that Navicat will generate by clicking on the "SQL Preview" tab:

Notice that the Type name is "Untitled" since we haven't yet saved the definition. That is expected.

Upon clicking on the "Save" button, we are presented with a "Save As" dialog where we can give our Type a name. Let's call it "film_rating":

Using the film_rating Type In a Table Definition

Now we can use the "film_rating" type just like any other PostgreSQL data type. For instance, we can set a table column to our custom type. We can even change the type on an existing table provided that its data values conform to our Enum value. In fact, changing a column's type from a generic VARCHAR to the stricter ENUM is an efficient way to quickly determine if a column contains invalid values.

If we open the "film" table in the Navicat Table Designer, we can set the "rating" column to our "film_rating" type by selecting "(Type)" from the "Type" drop-down and then setting the "Object Type" to "film_rating":

Also make sure that the "Collation" field is blank.

If the column doesn't contain any invalid values, we should be able to Save the table definition without any errors or warnings.

One of the advantages to setting a column type to an ENUM is that Navicat will provide a drop-down for choosing a value:

Conclusion

In today's blog, we created an Enumerated Type using Navicat Premium 17's Type tool and updated an existing table to utilize our custom type in order to constrain column values. Part 4 will proceed with the Range Type.

Composite Types

Welcome to the second installment of this series on creating custom data types in PostgreSQL using Navicat Premium 17. In part 1, we learned how to create a custom Domain for the free DVD Rental database. A Domain is a user-defined data type with constraints such as NOT NULL and CHECK. In today's blog, we'll create a Composite Type to return complex data from a user-defined function.

PostgreSQL Types Defined

Types are generated using the CREATE TYPE command. It creates a Composite Type that may be used in stored procedures and functions as the data types of input parameters as well as returned values.

PostgreSQL's CREATE TYPE supports four primary variations:

• Composite Types: Define composite data which combines two or more data types. allowing creation of complex, multi-field data types that can represent intricate data structures.
• Enumeration Types: Defined as a fixed set of predefined, named values, restricting input to only those specific options.
• Range Types: Representing continuous intervals between values, enabling sophisticated operations on contiguous data ranges like dates or numbers.
• Base Types: User-defined types may be created based on existing base types like int, varchar, or numeric. While there isn't a specific "Base Type" for user-defined types, new types are essentially extensions or constraints applied to these underlying PostgreSQL base types.

In the next few sections we'll explore Composite Types in more detail by creating a Type and using it in a function.

The CREATE TYPE Statement

All Types are created using the CREATE TYPE statement. Let's say that we wanted to have a function that returns several values about a film such as the film ID, title, and release_year. Here is the statement that creates a type named "film_summary":

CREATE TYPE film_summary AS (    film_id INT4,    title VARCHAR(255),    release_year CHAR(4));

Creating a Type in Navicat 17

Navicat Premium 17 and Navicat for PostgreSQL 17 both offer a GUI-based tools for generating types without having to know all of the exact syntax. You'll find it under "Others" in the main toolbar:

Next, we'll click on the arrow next to the "New Type" item in the "Objects" toolbar. That bring up the four different options for creating a type. Select the "Composite" item from the context menu:

That will bring up a grid in which we can enter the field details. Since the three fields which make up the "film_summary" Type already exist, we can bring up the "film" table in the Table Designer and copy the Type and Length data from there. Here are the three fields highlighted in red:

The grid will already have an empty row for the first field. Once we've entered its details, we can add a new row by clicking on "Add Member". Here is the completed grid:

Before clicking the "Save" button we can take a look at the statement that Navicat will generate by clicking on the "SQL Preview" tab:

Notice that the Type name is "Untitled" since we haven't yet saved the definition. That is expected.

Let's assign the name now. Clicking on the "Save" button brings up the "Save As" dialog where we can give our Type a name of "film_summary":

Using the film_summary Type In a Function

Now it's time to use the "film_summary" as the return type of a function. Like the Type creation, we'll use Navicat's GUI tool to do so. To access the Function Designer, click the "Function" button on the main toolbar followed by "New Function" on the "Objects" toolbar:

The editor will pre-populate most of the syntax for the CREATE FUNCTION for us; we just need to supply a few details like the function name, input parameters, return type, and function body. Here is the completed CREATE FUNCTION statement:

CREATE FUNCTION get_film_summary (f_id INT4)  RETURNS film_summaryAS $BODY$  SELECT     film_id,    title,    release_year  FROM    film  WHERE    film_id = f_id;$BODY$  LANGUAGE SQL VOLATILE;

Also be sure to set the language to "SQL".

Once we click the "Save" button, our function is ready to be used. The quickest and easiest way to try a function is to click the "Execute" button. That will bring up a prompt for use to supply a value for the "f_id" parameter:

The results should then appear in a new Result tab:

Conclusion

In today's blog, we created a Composite Type using Navicat Premium 17's Type tool and designed a function that returns our Type. Part 3 will continue with Enumeration Types.

Domains

Storing data in proper formats ensures data integrity, prevents errors, optimizes performance, and maintains consistency across systems by enforcing validation rules and enabling efficient data management. For these reasons, top tier relational databases like PostgreSQL offer a variety of data types. In addition, PostgreSQL enables custom data type creation via the "CREATE DOMAIN" and "CREATE TYPE" statements, allowing developers to extend data types for enhanced application-specific data validation, integrity, and consistency. In today's blog, we'll learn how to create a custom Domain for the free DVD Rental database using Navicat Premium 17. Part 2 will cover Types.

A Quick Comparison of CREATE DOMAIN and CREATE TYPE

While both the CREATE DOMAIN and CREATE TYPE statements may be employed to create user-defined data types, there are some key differences to be aware of:

• CREATE DOMAIN creates a user-defined data type with constraints such as NOT NULL, CHECK, etc.
• CREATE TYPE creates a composite type used in stored procedures as the data types of returned values.

Creating An Email Domain

Domains centralize constraint management by allowing you to define reusable validation rules across multiple tables, such as creating a standard constraint that prevents NULL values and trims whitespace for specific field types. Here's an example that creates a domain for email addresses with a validation check:

CREATE DOMAIN email AS VARCHAR(255)CHECK (  VALUE ~ '^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}$');-- Use in a table definitionCREATE TABLE customer_contacts (  customer_id INT,  contact_email email);

Navicat Premium 17 and Navicat for PostgreSQL 17 both offer a GUI-based tools for generating domains and types without having to know all of the exact syntax. You'll find both under "Others" in the main toolbar. (Both menu items are highlighted in red below):

The Domain tool includes four tabs: General, Checks, Comment, and SQL Preview.

General Attributes

All domains are based on an underlying type. In this case, it's VARCHAR. Once we select an Underlying Type Category of "Base Type", we can select "pg_catalog" and "varchar" from the two Underlying Type drop-downs. We'll also need to make sure that our VARCHAR has a Length of 255. Here is the General Tab with all of that information provided:

Checks

On the next tab, we can define one or more checks to perform when someone attempts to assign a value to our type. Our check will test the value against a RegEx (regular expression):

SQL Preview

At this point we can either proceed to Save the Domain, which will execute the generated CREATE DOMAIN statement, or we can click on the SQL Preview tab to view the statement before saving:

Notice that the Domain name is "Untitled" since we haven't yet saved the definition. That is normal.

Using the email Domain In a Table

The best way to confirm that our "email" Domain was created is to try it in a table. The "staff" table in the "dvdrental" database includes an email field. Currently, it's storing values as a VARCHAR without any validation checks. We can change the type to our Domain by selecting the "(Domain)" option from the Type drop-down in the Table Designer and then choosing "public" and "email" for the Object Type:

Once we save the table, attempting to change (or add) a value which is not a valid email address will result in a constraint violation:

Conclusion

By creating a custom Domain for the free dvdrental database, we saw how domains help centralize constraint management by allowing us to define reusable validation rules. In part 2, we'll create our own type using Navicat Premium 17's Type tool.

Migrating data between heterogeneous repositories - that is to say, where the source and target databases are of different database management systems from different providers - presents several challenges. In some cases, it is possible to connect to both databases simultaneously. However, there are times that it is simply not possible. When presented with such a dilemma, database practitioners have no choice but to populate tables from a dump file. Navicat can be of great help in that process. The Import Wizard allows you to import data to tables/collections from a variety of sources, including CSV, TXT, XML, DBF and more. Moreover, you can save your settings as a profile for future use or setting automation tasks. In today's blog, we'll use the Navicat Import Wizard to migrate data from the PostgreSQL "dvdrental" database to a MySQL 8 instance using the FREE Navicat Premium Lite 17.

For this tutorial, we will populate the film table in MySQL 8 using the PostgreSQL DAT file. Here is the table definition in the Table Designer:

To launch the Import Wizard, right-click the target table in the Navicat Navigation Pane (or Ctrl-Click in macOS) and select "Import Wizard..." from the context menu:

The first screen of the wizard is where we select the source file. Note that Lite edition only supports text-based files, such as TXT, CSV, XML and JSON. Although we have a .dat file, we can select the Text file option, which encompasses .txt, .csv, and .dat formats:

On the next screen we'll choose the DAT file. There is one file for each table. The one for the film table is named "3061.dat":

Next it's time to set the delimiters. Records are delimited using the Line Feed (LF) character, while columns are separated using the TAB character. There are no quotes around text values, so be sure to remove the double quote (") character from the "Text Qualifier" text box:

On the next screen, you'll find a few additional options. Here, we have to uncheck the "Field Name Row" box because the DAT file does not include the field names. We'll also need to change the Date Order to Year/Month/Day ("YMD") and replace the forward slash (/) delimiter with the dash (-) as the dates we will be importing are in a YYYY-MM-DD hh:mm:ss.ms, i.e. 2013-05-26 14:50:58.951, format:

We have the option of choosing an existing table or create a new one. Since we selected the target table when launching the Import Wizard, it should be displayed here:

The next step is to map the source fields to those in the destination table. Here we mustn't just assume that they will line up. A quick look at an entry in the DAT file reveals that the last_update and special_features columns are reversed:

5 African Egg A Fast-Paced Documentary of a Pastry Chef And a Dentist who must Pursue a Forensic Psychologist in The Gulf of Mexico 2006 1 6 2.99 130 22.99 G 2013-05-26 14:50:58.951 {"Deleted Scenes"} 'african':1 'chef':11 'dentist':14 'documentari':7 'egg':2 'fast':5 'fast-pac':4 'forens':19 'gulf':23 'mexico':25 'must':16 'pace':6 'pastri':10 'psychologist':20 'pursu':17

We can right-click (or Ctrl-Click in macOS) anywhere in the dialog and select "Direct Match All" from the context menu to quickly map the field to those of the target table. However, once that is done, we must manually choose the last_update and special_features columns from the Target field drop-downs to change their order:

Note that field 13 (f13) can be safely ignored.

For the Import Mode, we can either Append or Copy the records, since the table should be empty:

When migrating from one database type to another, there is a strong chance of encountering data conversion errors. For that reason, it's a good practice to deselect the Advanced "Use extended insert statements" box. Doing so causes Navicat to issue separate INSERT statements for each record rather than combine multiple rows using syntax such as:

INSERT INTO `film` VALUES   (1, 'African Egg', 'A Fast-Paced...'),   (2, 'Rumble Royale', 'A historical drama...'),   (3, 'Catherine the Great', 'A new take on...'),   etc...

Now, it's time to hit the Start button to kick off the import process.

As expected, there were a couple of errors (3 to be exact), but 1000 of 1003 rows were added to the target table!

Conclusion

Navicat's Import Wizard can dramatically cut down the amount of time spent on migrating data between heterogeneous repositories. It supports a wide range of inputs, including CSV, TXT, XML, DBF, ODBC Data Sources and more.

Interested in giving Navicat Premium Lite 17 a try? You can download it for free here. It's available for Windows, macOS, and Linux operating systems.

Migrating data between heterogeneous repositories - that is to say, where the source and target databases are of different database management systems from different providers - presents several challenges. In some cases, it is possible to connect to both databases simultaneously. However, there are times that it is simply not possible. When presented with such a dilemma, database practitioners have no choice but to populate tables from a dump file. Navicat can be of great help in that process. The Import Wizard allows you to import data to tables/collections from a variety of sources, including CSV, TXT, XML, DBF and more. Moreover, you can save your settings as a profile for future use or setting automation tasks. In today's blog, we'll use the Navicat Import Wizard to migrate data from the PostgreSQL "dvdrental" database to a MySQL 8 instance using the FREE Navicat Premium Lite 17.

For this tutorial, we will populate the film table in MySQL 8 using the PostgreSQL DAT file. Here is the table definition in the Table Designer:

To launch the Import Wizard, right-click the target table in the Navicat Navigation Pane (or Ctrl-Click in macOS) and select "Import Wizard..." from the context menu:

The first screen of the wizard is where we select the source file. Note that Lite edition only supports text-based files, such as TXT, CSV, XML and JSON. Although we have a .dat file, we can select the Text file option, which encompasses .txt, .csv, and .dat formats:

On the next screen we'll choose the DAT file. There is one file for each table. The one for the film table is named "3061.dat":

Next it's time to set the delimiters. Records are delimited using the Line Feed (LF) character, while columns are separated using the TAB character. There are no quotes around text values, so be sure to remove the double quote (") character from the "Text Qualifier" text box:

On the next screen, you'll find a few additional options. Here, we have to uncheck the "Field Name Row" box because the DAT file does not include the field names. We'll also need to change the Date Order to Year/Month/Day ("YMD") and replace the forward slash (/) delimiter with the dash (-) as the dates we will be importing are in a YYYY-MM-DD hh:mm:ss.ms, i.e. 2013-05-26 14:50:58.951, format:

We have the option of choosing an existing table or create a new one. Since we selected the target table when launching the Import Wizard, it should be displayed here:

The next step is to map the source fields to those in the destination table. Here we mustn't just assume that they will line up. A quick look at an entry in the DAT file reveals that the last_update and special_features columns are reversed:

5 African Egg A Fast-Paced Documentary of a Pastry Chef And a Dentist who must Pursue a Forensic Psychologist in The Gulf of Mexico 2006 1 6 2.99 130 22.99 G 2013-05-26 14:50:58.951 {"Deleted Scenes"} 'african':1 'chef':11 'dentist':14 'documentari':7 'egg':2 'fast':5 'fast-pac':4 'forens':19 'gulf':23 'mexico':25 'must':16 'pace':6 'pastri':10 'psychologist':20 'pursu':17

We can right-click (or Ctrl-Click in macOS) anywhere in the dialog and select "Direct Match All" from the context menu to quickly map the field to those of the target table. However, once that is done, we must manually choose the last_update and special_features columns from the Target field drop-downs to change their order:

Note that field 13 (f13) can be safely ignored.

For the Import Mode, we can either Append or Copy the records, since the table should be empty:

When migrating from one database type to another, there is a strong chance of encountering data conversion errors. For that reason, it's a good practice to deselect the Advanced "Use extended insert statements" box. Doing so causes Navicat to issue separate INSERT statements for each record rather than combine multiple rows using syntax such as:

INSERT INTO `film` VALUES   (1, 'African Egg', 'A Fast-Paced...'),   (2, 'Rumble Royale', 'A historical drama...'),   (3, 'Catherine the Great', 'A new take on...'),   etc...

Now, it's time to hit the Start button to kick off the import process.

As expected, there were a couple of errors (3 to be exact), but 1000 of 1003 rows were added to the target table!

Conclusion

Navicat's Import Wizard can dramatically cut down the amount of time spent on migrating data between heterogeneous repositories. It supports a wide range of inputs, including CSV, TXT, XML, DBF, ODBC Data Sources and more.

Interested in giving Navicat Premium Lite 17 a try? You can download it for free here. It's available for Windows, macOS, and Linux operating systems.

Navicat's database administration and development tools have long been designed with collaboration in mind. Now, thanks to the recent launch of Navicat On-Prem Server, collaboration takes center stage, allowing us to share connection settings, queries, aggregation pipelines, snippets, model workspaces, Business Intelligence (BI) workspaces and virtual group information with team members across the globe - and all in real-time. While the last several blogs described how to share database objects using Navicat On-Prem Server, today's entry will focus on how to accomplish the same thing using Navicat Premium 17.

Viewing Navicat On-Prem Server Objects In Navicat Premium 17

When you are logged into Navicat On-Prem Server, you'll be able to see the On-Prem Server in the Navigation Pane of Navicat database administration and development tools. Shared objects behave exactly like local ones and can be viewed, edited, and deleted directly in Navicat.

Another way to see On-Prem Server Objects is by clicking on your user icon in the upper-right corner of the main Navicat window. Doing so brings up the Manage Cloud dialog. It shows the name of the On-Prem Server, the Host IP, usage details, the number of projects created, and more!

If we wish, we can access the On-Prem Server by clicking on the "Manage Account" link. It will open the On-Prem Server in a new browser tab.

Creating a New Object

Sharing is not limited to pre-existing objects. We can also create a new object for the project we want to associate it with. Let's start by creating a new code snippet.

First, we'll open a query under the "DVDRental MySQL DB" project. From there, we'll create a new code snippet just as we always would. Here is a code snippet named "customer - payment table join" in Navicat Premium:

After we save our new code snippet, it's instantaneously shared with all project members! We can verify this by opening the On-Prem Server in a browser and clicking on "Snippets":

Sharing Existing Objects

Sharing an existing object is just as easy. For example, we can share any Navigation Pane object simply by dragging it from a database instance to the same section under the On-Prem Server, i.e. from Queries to Queries.

For other objects that don't appear in the Navigation Pane, such as Business Intelligence (BI) or Model workspaces, we can copy from the Objects pane of the local instance:

Next, we paste the copied workspace into the Objects pane after selecting a project under the On-Prem Server in the Navigation Pane:

Again, we can verify that the workspace has been shared with team members by clicking on "BI Workspaces" in Navicat On-Prem Server:

Conclusion

In today's blog, we learned how Navicat Premium 17 works with Navicat On-Prem Server to take collaboration to a whole new level. Navicat 17 allow us to share connection settings, queries, aggregation pipelines, snippets, model workspaces, Business Intelligence (BI) workspaces and virtual group information with team members across the globe. Meanwhile, the On-Prem Server also offers a comprehensive suite of administration and development tools!

You can download Navicat Premium 17 for a 14-day fully functional FREE trial. On-Prem Server may be downloaded here. Both are available for Windows, macOS, and Linux operating systems.

One of the standout features of PostgreSQL is its extensive support for user-defined functions and data types. This allows developers to create custom conversion, operator, and aggregate functions. Aggregates offer a powerful way to perform complex calculations and transformations on data, going beyond the standard SQL aggregate functions like SUM, AVG, and COUNT. Both Navicat for PostgreSQL and Navicat Premium make it easy to write custom functions and aggregates that integrate seamlessly with the database, thanks to their specialized graphical user interface (GUI). All we need to do is provide a few details and Navicat produces the pgSQL statement for us! In today's blog, we'll be creating an aggregate to work with the DVD Rental database that concatenates movie titles by category.

About Aggregates

Aggregates are a fundamental feature of SQL that allow you to perform calculations or transformations on a set of rows and return a single result. The most common aggregate functions are SUM, AVG, COUNT, MIN, and MAX, which allow you to quickly summarize data by calculating totals, averages, counts, minimum values, and maximum values, respectively.

However, the built-in aggregate functions provided by SQL don't always meet the specific needs of an application. This is where the ability to create custom aggregates becomes useful. Custom aggregates allow you to define your own logic for summarizing and transforming data, going beyond the standard set of SQL aggregates. The process typically involves defining a state transition function, which is called for each row to update an accumulator, as well as an optional final function that is called to produce the final aggregate result.

Generating the Transition and Final Functions

Our transition function, array_append_state(), will be called for each row to update the aggregate state.

To access Navicat's function editor, click the Function button in the main button bar and then click on "New Function" in the Objects toolbar:

Navicat will start us off with the main function definition. From there, we'll supply the function name, input parameters, and body:

CREATE FUNCTION "public"."array_append_state" (current_state text[], new_value text)  RETURNS text[] AS $BODY$BEGIN  RETURN array_append(current_state, new_value);END$BODY$  LANGUAGE 'plpgsql' VOLATILE;

When we're done, we can click Save to create the function.

Now we'll go back to the Objects tab and click on "New Function" to create the final function.

The array_to_comma_string() function will take an array of film titles and insert a comma between each element:

CREATE FUNCTION "public"."array_to_comma_string" (state text[])  RETURNS text AS $BODY$BEGIN  RETURN array_append(state, ', ');END$BODY$  LANGUAGE 'plpgsql' VOLATILE;

Creating the comma_concat() Aggregate Function

We can now plug our two functions into Navicat's Aggregate Editor. We can access the editor by clicking the Others button in the main button bar and then selecting "Aggregate" from the context menu:

In the form, we'll set the Input type to "text", enter a State type of "text[]" and supply our State and Final functions. Also, make sure that the Initial condition is an empty array ("{}"):

We can see the generated SQL by clicking on the Preview tab:

CREATE AGGREGATE "public"."Untitled" (In "pg_catalog"."text")(  SFUNC = "public"."array_append_state",  STYPE = "pg_catalog"."text[]",  FINALFUNC = "public"."array_to_comma_string",  INITCOND = "{}",  PARALLEL = UNSAFE);ALTER AGGREGATE "public"."Untitled"("pg_catalog"."text") OWNER TO "postgres";

Notice that the name of the aggregate is "Untitled". Navicat will prompt us for the name when we hit the Save button and execute the command with the name that we provide.

Using Our Custom Aggregate

We can now invoke our aggregate function just like any other function. Here's a query that fetches a list of movies by category:

SELECT     c.name AS category,    comma_concat(f.title) AS moviesFROM category cJOIN film_category fc ON c.category_id = fc.category_idJOIN film f ON fc.film_id = f.film_idGROUP BY c.nameORDER BY c.name;

Conclusion

In today's blog, we created a custom PostgreSQL aggregate in Navicat Premium to work with the DVD Rental database that concatenates movie titles by category.

Interested in giving Navicat Premium 17 a try? You can download it for a 14-day fully functional FREE trial. It's available for Windows, macOS, and Linux operating systems.

Several recent blogs have been dedicated to Navicat's latest collaboration tool: Navicat On-Prem Server. It's an on-premise solution for hosting a cloud environment that allows you to synchronize your connection settings, queries, aggregation pipelines, snippets, model workspaces, BI workspaces and virtual group information across all your devices. In today's blog, we'll learn how to develop queries directly in On-Prem Server, and then share them with our team in real-time.

Opening a Connection and Database

In the last blog we created the "dvdrental MySQL 8" Connection and associated it to the "DVDRrental MySQL DB" project:

Clicking the connection name opens a new browser tab where we can login to the database:

Click the "Open Connection" button to establish a secure connection to the database instance.

On the Connection screen, you'll find a list of databases under the Connection Tree on the left-hand side:

Double-clicking a database opens it and expands the tree to show tables, views, functions, events, and queries:

Writing and Saving a Query

Now we'll write a brand new query in On-Prem Server's query editor. There are a couple of options for getting there:

First, we'll click on the "Queries" item under "Connection Tree" to select it. From there, we can either:

• click on the ellipsis (...) that appears beside the item and select "New Query" from the popup menu, or
• click on the "New Query" button at the top of the main Query screen.

Both options are highlighted in red below:

Performing either of the above actions will open a new browser tab where we can develop our query.

Here's a query that fetches all films with a rental rate of 99 cents:

To save the query, click the "Save" icon at the top of the screen. A dialog will appear where we can enter the name:

Confirming That a Query Has Been Added to a Project

After clicking the "Save" button, if we return to the "Connection" tab and refresh the list of queries, we should see our new query:

We should also see that the "DVDRrental MySQL DB" project now shows the queries icon:

Now all members of the "DVDRrental MySQL DB" project will have access to the "Number of movies with rental rate of 0.99" query according to their assigned user rights.

Managing Project Members

If you wish to modify a project's members, you can click the ellipsis next to the project name on the "All Projects" screen and select the "Manage Members" option from the context menu:

That will present a dialog where you can select members from a list:

If the user you are looking for does not appear in the list, you can add them via the Advanced Configurations -> Organisation Account -> All Users screen. That's where you can manage all of the users within your organisation. Navicat On-Prem Server allows creating local users, or creating external users using LDAP or AD authentication.

Conclusion

Today's blog covered how to develop queries directly in the Navicat On-Prem Server, which we then shared with our team in real-time.

Interested in giving Navicat On-Prem Server a try? You can download it for a 14-day fully functional FREE trial. It's available for Windows, macOS (using Homebrew), and Linux operating systems.

Adding Projects, Members, and Connections

In a recent blog, we learned about Navicat's latest collaboration tool: Navicat On-Prem Server. It's an on-premise solution for hosting a cloud environment where you can securely store Navicat objects internally at your location. Today's blog will cover how to work with projects in Navicat On-Prem Server. Topics covered will include how to create a project, adding members, and how to configure a connection to your database instance(s).

Creating a New Project

The first time you launch On-Prem Server, the main window will indicate that you have no projects yet. To add one, click the "+New" button and select "New Project" from the popup menu:

Enter the project name in the dialog and click "Create" to add the new project.

After the dialog closes, you should see your new project under "Projects" in the main window:

Adding Members To a Project

Our new project will now allow us to share queries, code snippets, virtual group information, as well as model and Business Intelligence (BI) workspaces with our team members. Let's add some team members to the project now. There are a couple of ways to do that; the first is to click on the ellipsis (...) to the right of the project name. That will open a submenu with additional commands. The one we want is "Manage Members":

Clicking it opens a dialog where we can add members.

The other way to access the dialog is to click on the project name. That will show the project details. There, we can click the "Manage Members" icon:

When the Collaborate dialog first opens, only the project owner will appear in the list. To add members, click the "+Add Member" button at the bottom of the dialog:

That will show some additional fields to the bottom of the dialog where we can enter the member's name/email and rights. These include:

• Can Manage & Edit: Read Objects, Write Objects, Manage Members and Rename Projects
• Can Edit: Read Objects and Write Objects
• Can View: Read Objects

Click "Add" to append the member to the list.

Preparing the Project For Object Sharing

Before we can share queries, code snippets, virtual group information, or model workspaces and Business Intelligence (BI) workspaces with our team members, we must first create a connection to the database instance.

The first step for creating a new connection is similar to that of creating a new project. Again, we will click the "+New" button, but this time, we will choose the "New Connection" command from the popup menu:

The New Connection dialog will guide us through the process.

The first step is to select the connection type - i.e. what type of database are we connecting to:

On the next screen, we can provide the Connection Name, Project, as well as the authentication details:

Once all of the fields have been populated, we can test the connection by clicking the "Test Connection" button. We can then proceed to create the new connection by clicking on "New".

Once the dialog closes, we should see the new connection on the Connections screen:

Conclusion

In today's blog we learned how to create a project, adding members, and how to configure a connection to your database instance(s) in Navicat On-Prem Server. Now that we've configured the connection to our database instance, next week we'll proceed to establish a connection to the database, and share some Navicat objects with our team members via the project that we created here today.

Available in all editions of Navicat database administration and development tools (including Navicat Premium Lite!), Code Snippets allows you to insert reusable code into your SQL statements when working in the SQL Editor. Besides gaining access to a collection of built-in snippets, you can also define your own. One of the built-in categories supplies special Navicat-only Syntax for customizing the query results tab name as well as for supplying runtime parameters. Today's blog will demonstrate how to use both snippets in your queries using the free MySQL Sakila Database and Navicat Premium 17.

About the Code Snippet Pane

Located on the right-hand side of the SQL Editor, the Code Snippets Pane provides an easy way to insert reusable code into SQL statements when working in the SQL Editor. If the editor window is docked to the Navicat main window, you can click the () icon in the Information pane to view the snippets library.

You can bring up the two code snippets that we'll be learning about today by selecting the "Navicat-only Syntax" item in the Code-Snippets drop-down menu:

Customizing the Result Tab Name

Every result set generated by a query is displayed in a separate tab below the Query Editor. Each tab is given the name "Result n" by default where "n" is the order in which the query was executed. For example, the first query will be named "Result 1", the second "Result 2", etc.

Clicking the "Customize Result Tab Name" snippet will insert some special Navicat-only syntax at the current cursor position in the editor:

Once we've replaced the "tab_name" text with the desired tab name and the "Statement..." placeholder with the SQL, executing the query will now display the results with the name that we specified:

There are a couple of other ways to insert a Code Snippet in the Query Editor. We can:

• drag and drop a snippet from the library into the editor, or
• start typing the name of a snippet in the editor. Smart code completion will pop up a list of suggestionsfor the word completion automatically. From there, we can select the snippet from the list to insert to the codeinto the editor.

  

Supplying a Runtime Parameter

One of the advantages of using a stored procedure is that you can supply one or more input parameters rather than hard-code values. Thanks to Navicat's Runtime Parameter Code Snippet, you can achieve the same result using regular SELECT statements.

Clicking the "Runtime Parameter" Snippet will insert a placeholder for the parameter at the current cursor position in the editor:

Now, when we execute the query, Navicat will present an input parameter dialog for us to provide the value to use:

Conclusion

In today's blog we learned how to use Navicat-only Syntax Code Snippets for customizing the query results tab name as well as for supplying runtime parameters to our queries. These were executed in Navicat Premium 17. Interested in giving Navicat Premium 17 a try? You can download it for a 14-day fully functional FREE trial. It's available for Windows, macOS, and Linux operating systems.

Customizing the App Server Settings and Setting Up Notifications

Navicat On-Prem Server is the perfect on-premise cloud solution for organizations who wish to maintain complete control over their data. By maintaining full control over your system, you can ensure 100% data privacy, while still benefiting from the convenience and features of a cloud-based solution. Navicat On-Prem Server also provides comprehensive management tools for MySQL and MariaDB, enabling efficient administration, monitoring, and management of your database instances.

Last week's blog article covered the first steps for configuring Navicat On-Prem Server on Windows 10 with a MySQL 8 Community Server instance. Today we will go over the App Server, Notification Settings, and Confirmation steps to complete the configuration process.

Customizing the App Server Settings

As mentioned in part 1, after you have installed Navicat On-Prem Server and start it for the first time, a browser will pop up and open the Welcome page of your Navicat On-Prem Server at http://:. The host address is the host name of the system that installed Navicat On-Prem Server, and the port number is 3030 by default. Hence, the server URL will usually be "http://127.0.0.1:3030/".

We can specify our own Application Port, Web URL, and Binding IP Address at this point. You may want to specify a Binding IP Address for users to access Navicat On-Prem Server if the machine has been assigned multiple IP addresses. An address of "0.0.0.0" means all IPv4 addresses on the machine, while "::" means all IPv4 and IPv6 addresses on the machine.

We'll stick with the defaults for the purposes of this tutorial.

Click Next > to continue.

Set Up Notifications

Whenever an event, such as two-step verification, security activities, new device sign-in, project invitation, or a system problem is raised while you are using it, Navicat On-Prem Server can send out notifications via SMS text message and/or email.

Supported SMS service providers include Clickatell, Twilio or Others.

Email settings include the SMTP Server, Port, login details, Sending Address, and a few other options:

Click Next > to continue.

Confirmation

The confirmation screen gives you a chance to review all of your information before finalizing the configuration process.

Here, you can either click the Back button to change some details or Finish to proceed. Note that the initial configuration process may take a few minutes for setting up the repository database.

After the configuration has completed successfully, a login page will be displayed and you can log in Navicat On-Prem Server with the manager user account.

Conclusion

This two-part series walked us through the steps to configure Navicat On-Prem Server on Windows 10 with a MySQL 8 Community Server instance.

Interested in giving Navicat On-Prem Server a try? You can download it for a 14-day fully functional FREE trial. It's available for Windows, macOS (using Homebrew), and Linux operating systems.

Configuring the Superuser, On-Prem Server Profile, and Repository Server

Navicat On-Prem Server is an on-premise solution for hosting a cloud environment where you can securely store Navicat objects internally at your location. It's one of two Navicat products whose goal is to foster increased collaboration amongst team members - the other being Navicat Cloud. The main difference between these two solutions is the location of shared objects: in the case of Navicat Cloud, they are stored in a central location on Navicat's servers, whereas the Navicat On-Prem Server resides on your organization's infrastructure. That being said, you can also install Navicat On-Prem Server on Amazon Linux 2 or within a Docker container. Today's blog will cover the first steps for configuring Navicat On-Prem Server on Windows 10 with a MySQL 8 Community Server instance. The next blog article will complete the series with the remaining steps.

Starting and Stopping Navicat On-Prem Server

After the installation, Navicat On-Prem Server starts automatically. You can configure this behavior via the taskbar icon:

Right-clicking the icon will open the context menu. There, you can start & stop Navicat On-Prem Server as well as enable or disable the Auto Start feature:

The Welcome Page

After you have installed Navicat On-Prem Server and start it for the first time, a browser will pop up and open the Welcome page of your Navicat On-Prem Server at http://<your_ip_address>:<port_number>. The host address is the host name of the system that installed Navicat On-Prem Server, and the port number is 3030 by default. Hence, the server URL will usually be "http://127.0.0.1:3030/".

On the Welcome page, we can either click the Setup On-Prem Server button to complete the basic configuration of Navicat On-Prem Server or import existing settings if we already have a Navicat On-Prem Server.

We'll enter the configuration details manually since this is our first installation. There are six sections to fill in:

• Superuser
• On-Prem Server Profile
• Connect to Repository Server
• App Server
• Notification Settings
• Confirmation

The next several sections will cover the first three points above.

Create the Superuser Account

Superuser is a local user (Admin) account which has unlimited access to Navicat On-Prem Server functionalities. You may supply the following profile information of the superuser: Username, Password, Full Name, Email, Mobile Number,Preferred Language and Appearance. You can also upload a profile photo:

Once you've entered all of the details, click Next> to continue.

Set the On-Prem Server Profile

On the next page, you can provide a few details about the On-Prem Server such as the as the On-Prem Server Name and Company Name. You may upload a server logo image as well:

Click Next> to move on to the next screen.

Connect to the Repository Server

The repository database stores all user information and Navicat objects. Supported databases include:

• MySQL
• MariaDB
• PostgreSQL
• SQL Server
• Amazon RDS

Ideally you should allocate a separate instance to act as the repository database. Moreover, it should not reside on a production server.

For greater security, you can use SSL authentication and specify the type of encryption cipher to use. Many of the common cipher suites are supported, including TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, and others.

Going Forward

In Part 1 of Getting Started with Navicat On-Prem Server, we went over how to configure the Superuser, On-Prem Server Profile, and Repository Server on Windows 10 with a MySQL 8 Community Server instance. The conclusion will cover the App Server, Notification Settions, and Confirmation.

Interested in giving Navicat On-Prem Server a try? You can download it for a 14-day fully functional FREE trial. It's available for Windows, macOS (using Homebrew), and Linux operating systems.

Navicat Collaboration provides the means for your team to collaborate on a variety of database objects, including connection settings, queries, aggregation pipelines, snippets, model workspaces, BI workspaces and virtual group information. Navicat offers two options for Collaboration: Navicat Cloud and Navicat On-Prem Server. Whereas Navicat Cloud offers a central space for your team to store Navicat objects, Navicat On-Prem Server is an on-premise solution for hosting a cloud environment where you can securely store Navicat objects internally at your location. Today's blog will describe how Navicat On-Prem Server helps foster collaboration within your team and manage MySQL and MariaDB instances more effectively.

Navicat On-Prem Server At a Glance

Navicat On-Prem Server is an on-premise solution that lets you host a private cloud environment for managing Navicat files internally, enabling distributed teams to collaborate in real time, share data, coordinate tasks, and communicate seamlessly through a centralized platform.

With Navicat On-Prem Server, you retain full control over your system and guarantee complete data privacy, while still enjoying the convenience and features of a cloud-based solution. It also provides powerful management tools for MySQL and MariaDB databases, allowing you to easily administer, monitor, and manage your databases efficiently.

Collaboration Tools

Using Navicat On-Prem Server, you can synchronize your connection settings, queries, aggregation pipelines, snippets, model workspaces, BI workspaces, and virtual group information across all your devices. You can also invite your colleagues to join the project, enabling them to create and edit files collaboratively so you can work together in real-time from anywhere across the globe.

Using the Object Filter, you can locate the specific objects you need quickly through vast amounts of content, ensuring you can always find the objects that matter most, in a timely fashion.

You can assign a role to coworkers for each project they work on and grant them access to projects based on their business functions. Each role determines whether a team member can create, view, and modify project files.

All collaborative activities are tracked in real time within the Activity Log, allowing you to easily monitor everything happening in your project. You can also review the recent actions of specific team members, giving everyone a clear view of what others are working on.

MySQL and MariaDB Management

Navicat On-Prem Server comes with a comprehensive set of tools for efficiently managing core database objects, allowing you to:

• administer users, permissions, and privileges
• browse and manage tables, views, events, functions, and stored procedures
• write, execute, and analyze database queries via the built-in SQL editor

MySQL and MariaDB connections are managed through an intuitive, user-friendly interface. Its simple design allows you to perform essential database operations with ease. Advanced search and filtering tools help you quickly find and work with specific database objects.

The clear and responsive interface breaks down query writing into structured tabs - perfect for managing a variety of database objects, including tables, views, events and functions.

Navicat On-Prem Server features both a grid-style interface and detailed form view for viewing, updating, and deleting records. There are also specialized cell editors for text, hexadecimal, images, and web content. There are also advanced sorting and filtering options to further enhance your data management capabilities.

Many of the same features found in Navicat development and administration tools are also included in Navicat On-Prem Server to help accelerate the coding process. These include syntax highlighting, query explain, query result pinning, along with SQL stats such as query execution time.

Conclusion

This blog described how Navicat On-Prem Server helps foster collaboration within your team as well as more effectively manage MySQL and MariaDB instances. Some of the other features that you'll find in Navicat On-Prem Server include SMS and email notification, LDAP/AD, web-based interface, link sharing, firewall security and more!

Interested in giving Navicat On-Prem Server a try? You can download it for a 14-day fully functional FREE trial. It's available for Windows, macOS (using Homebrew), and Linux operating systems.

One of the most powerful SQL features is the JOIN operation, providing an elegant and simple means of combining every row from one table with every row from another table. However, there are times that we may want to find values from one table that are NOT present in another table. As we'll see in today's blog article, joins can be utilized for this purpose as well, by including a predicate on which to join the tables. Known as anti joins, these can be helpful in answering a variety of business-related questions, such as:

• Which customers did not place an order?
• Which employees have not been assigned a department?
• Which salespeople did not close a deal this week?

This blog will offer a primer on the types of anti joins and how to write them using a few examples based on the PostgreSQL dvdrental database. We'll write and execute the queries in Navicat Premium Lite 17.

Two Types of Anti Joins

There are two types of anti joins:

• left anti join: returns rows in the left table that have no matching rows in the right table
• right anti join: returns rows in the right table that have no matching rows in the left table

Returned rows are shown in blue in the diagram below:

The next section will walk through a few different syntaxes we can use to create an anti join, using a left anti join for examples.

Left Anti Join Using EXISTS

Let's say that we wanted to find all the actors in the dvdrental database that didn't appear in any film. Unfortunately, SQL doesn't have a built-in syntax for this operation, but we can emulate it using EXISTS, or, more specifically, NOT EXISTS. Here's what that query would look like:

SELECT *FROM actor aWHERE NOT EXISTS (  SELECT * FROM film_actor fa  WHERE a.actor_id = fa.actor_id)

If we run it in Navicat Premium Lite 17, we get the following results:

Beware of NOT IN!

Since EXISTS and IN are equivalent, you might be tempted to conclude that NOT EXISTS and NOT IN are also equivalent, but this is not always the case! They are only equivalent if the right table (in this instance, film_actor) has a NOT NULL constraint on the foreign key (the actor_id).

In this specific instance, the NOT IN query returns the same results because of the NOT NULL constraint on the actor_id column:

If the actor_id column did allow nulls, an empty result set would be returned. We can verify this via the following query:

SELECT *FROM actorWHERE actor_id NOT IN (1, 2, 3, 4, 5, NULL)

The above query doesn't return any rows because NULL represents an UNKNOWN value in SQL. Since we cannot be sure whether actor_id is in a set of values of which one value is UNKNOWN, the whole predicate becomes UNKNOWN!

The easiest way to avoid the danger posed by the NOT IN syntax is to stick with NOT EXISTS. It's really not even worth gambling on the presence of a NOT NULL constraint as the DBA might temporarily turn off the constraint to load some data, rendering your query useless in the interim.

Alternate Syntax

As alluded to in the introduction, it's also possible to perform an Anti Join using LEFT and RIGHT JOINs. For that to work, you need to add a WHERE clause with the IS NULL predicate. Here's the LEFT JOIN version of that syntax:

SELECT a.*FROM actor a  LEFT JOIN film_actor fa  ON a.actor_id = fa.actor_idWHERE fa.actor_id IS NULL

Just be aware that the LEFT/RIGHT JOIN syntax may run more slowly because the query optimizer doesn't recognize this as an ANTI JOIN operation.

Conclusion

In today's blog we learned how to emulate a Left Anti Join using three variations of SQL syntax. Of these, NOT EXISTS should be your first choice as it best communicates the intent of an ANTI JOIN and tends to execute the fastest.

Interested in giving Navicat Premium Lite 17 a try? You can download it for a 14-day fully functional FREE trial. It's available for Windows, macOS, and Linux operating systems.

Most database developers and administrators are familiar with the standard inner, outer, left, and right JOIN types. While these can be written using ANSI SQL, there are other types of joins that are based on relational algebra operators that don't have a syntax representation in SQL. Today we'll be looking at one such join type: the Semi Join. Next week we'll tackle the similar Anti Join. To gain a better understanding of how these types of joins work, we'll execute some SELECT queries in Navicat Premium Lite 17 against the PostgreSQL dvdrental database. It's a free database that's based on the MySQL Sakila Sample Database.

Semi Joins Explained

Imagine for a moment that ANSI SQL did support Semi Joins. If it did, the syntax would probably be similar to that of the Cloudera Impala syntax extension, which is LEFT SEMI JOIN and RIGHT SEMI JOIN. With that in mind, here's what a query that utilizes a Semi Join might look like:

SELECT *FROM actorLEFT SEMI JOIN film_actor USING (actor_id)

The above query would return all actors that played in films. That catch is that we don't want any films in the results, nor do we want multiple rows of the same actor. We only want each actor once (or zero times) in the result. The word "Semi" originates from Latin and translates to "half" in English. Hence, our query implements only "half the join", in this case, the left half. In SQL, there are two alternative syntaxes that we can use to accomplish a Semi Join: EXISTS and IN.

Semi Joins Using EXISTS

Here is the equivalent of the Semi Join using EXISTS:

SELECT *FROM actor aWHERE EXISTS (  SELECT *   FROM film_actor fa  WHERE a.actor_id = fa.actor_id)

If we execute our query in Navicat Premium Lite 17, we can see that it works just as expected:

Rather than use a join, the EXISTS operator checks for the presence of one or more rows for each actor in the film_actor table. Thanks to the WHERE clause, most databases will be able to recognize that we're performing a SEMI JOIN rather than an ordinary EXISTS() predicate.

Semi Joins Using IN

IN and EXISTS are exactly equivalent SEMI JOIN emulations, so the following query will produce the exact same results in most databases as the previous EXISTS query:

SELECT *FROM actorWHERE actor_id IN (  SELECT actor_id FROM film_actor)

Here again is the above query and results in Navicat Premium Lite 17:

EXISTS is considered to be the more powerful (albeit a bit more verbose) syntax.

Conclusion

In today's blog we learned how to emulate a Semi Join using ANSI SQL syntax. In addition to being the optimal solution in terms of "correctness", there are also some performance benefits when using a "SEMI" JOIN rather than an INNER JOIN, as the database can stop looking for matches as soon as it found the first.

Interested in giving Navicat Premium Lite 17 a try? You can download it for a 14-day fully functional FREE trial. It's available for Windows, macOS, and Linux operating systems.

If you have been writing SQL queries for some time, you are probably quite familiar with the WHERE clause. While it has no effect on aggregated fields, there is a way to filter records according to aggregate values, and that is by using the HAVING clause. This blog will cover how it works as well as provide a few examples on using it in SELECT queries.

Aggregation and the HAVING Clause

Aggregation is typically used in conjunction with grouping. In SQL, that's accomplished using the GROUP BY clause. Aggregation, together with grouping, allows us to glean high level insights into our data. For example, an eCommerce company might want to track sales over a given time period.

In many cases, we may not want to apply the GROUP BY clause on the entire dataset. In those instances, we can employ the GROUP BY command along with the conditional HAVING clause to filter out unwanted results. Similar to the WHERE clause, HAVING specifies one or more filter conditions, but for a group or an aggregation. As such, HAVING is always placed after the WHERE and GROUP BY clauses but before the (optional) ORDER BY clause:

SELECT column_listFROM table_nameWHERE where_conditionsGROUP BY column_listHAVING having_conditionsORDER BY order_expression

Some Practical Examples

To get a better idea on how HAVING works, let's run a few SELECT queries against the Sakila Sample Database.

Our first query lists our top movie renters, sorted in descending order, so that the person with the most rentals appears at the top. We'll use the HAVING clause to remove customers with less than three rentals in order to shorten the list somewhat:

SELECT  c.customer_id,  c.first_name,  c.last_name,  COUNT(r.rental_id) AS total_rentalsFROM   customer AS c    LEFT JOIN rental AS r ON c.customer_id = r.customer_idGROUP BY c.customer_idHAVING total_rentals >= 3ORDER BY total_rentals DESC;

Here is the query and the first page of results in Navicat Premium:

Judging by those rental numbers, we could have narrowed down the list substantially more!

Filtering Rows Using Both WHERE and HAVING

Just as GROUP BY and ORDER BY are applied at different points in the querying process, so too are WHERE and HAVING. Hence, we can include both to filter results both before and after grouping and aggregation. For example, we can add a WHERE clause to restrict results to the first half of a given year:

SELECT  c.customer_id,  c.first_name,  c.last_name,  COUNT(r.rental_id) AS total_rentalsFROM   customer AS c    LEFT JOIN rental AS r ON c.customer_id = r.customer_idWHERE r.rental_date BETWEEN '2005-01-01' AND '2005-06-30'GROUP BY c.customer_idHAVING total_rentals >= 3ORDER BY total_rentals DESC;

Once again, here is the above query and the first page of results in Navicat Premium:

Combining Multiple Conditions

Just as the WHERE clause supports multiple conditions using the AND and OR keywords, so too does HAVING. For example, we could find customers whose rental numbers fall within a given range by modifying the HAVING clause to something like the following:

HAVING total_rentals >= 3 AND total_rentals <= 10

Conclusion

In today's blog we learned how to filter grouped and aggregated fields using the HAVING clause.

Interested in Navicat Premium? You can try it for 14 days completely free of charge for evaluation purposes!

The SQL EXISTS operator offers us an easy way to retrieve data based on the existence (or non-existence) of some other data. More specifically, it's a logical operator that evaluates the results of a subquery and returns a boolean value indicating whether rows were returned or not. While the IN operator can be utilized for much the same purpose, there are some differences to be aware of. Today's blog will cover how to use the EXISTS operator using a few examples as well as provide some guidance as to when to use EXISTS rather than IN.

EXISTS In Action

Although the EXISTS operator can be used in a SELECT, UPDATE, INSERT or DELETE statement, we'll stick with SELECT queries to keep things simple. As such, the syntax we will be using will closely resemble this:

SELECT column_name(s) FROM table_nameWHERE EXISTS ( SELECT column_name(s)                FROM table_name               WHERE condition );

We'll be executing our queries against a couple of PostgreSQL tables - customer and account - such as those we might find in a banking database. Here they are in Navicat for PostgreSQL's Grid View:

Now we can see all the customers who have an account associated with their customer_id using the following query:

SELECT *FROM customer CWHERE EXISTS ( SELECT *               FROM account A               WHERE C.customer_id = A.customer_id );

Here is the above query with the results in the Navicat Premium's Query Editor:

Using NOT with EXISTS

Conversely, prefacing the EXISTS operator with the NOT keyword causes the query to only select records where there is no matching row in the subquery. We can use NOT EXISTS to fetch all orphaned accounts, that is to say, accounts with no associated customer:

SELECT *FROM account AWHERE NOT EXISTS ( SELECT *                   FROM customer C                   WHERE A.customer_id = C.customer_id );

That returns the account for customer #4 since there is no customer with that ID in the customer table.

Replacing EXISTS with Joins

Queries that use the EXISTS operator can be a little slow to execute because the subquery needs to be executed for each row of the outer Query. For that reason, you should consider using joins whenever possible. In fact, we can rewrite the above EXISTS query using a LEFT JOIN:

SELECT C.*FROM customer C  LEFT JOIN account A ON C.customer_id = A.customer_id;

IN vs EXISTS Operators

Although the IN operator is typically used to filter a column for a certain list of values, it can also be applied to the results of a subquery. Here's the equivalent to our first query, this time using IN rather than EXISTS:

SELECT * FROM customer WHERE customer_id IN (SELECT customer_id FROM account);

Note that we can only select the column that we want to compare against, as opposed to SELECT *. Nonetheless, the IN query produces the same results:

With both operators being so similar, database developers are often unsure as to which to use. As a general rule, you should use the IN operator when you want to filter rows based on a specific list of values. Use EXISTS when you want to check for the existence of rows that meet certain conditions in a subquery.

Conclusion

In today's blog we learned how to use the EXISTS operator as well as how to decide whether to use EXISTS or IN.

Interested in giving Navicat Premium 17 a try? You can download it for a 14-day fully functional FREE trial. It's available for Windows, macOS, and Linux operating systems.

In the mid nineties, Sun Microsystems came out with a language that you could "write once, [and] run everywhere." That language was, of course, Java. And, while it did go on to be one of the most popular programming languages until this day, their slogan turned out to be just a little optimistic. The course of the Java language does bear some strong similarities to that of SQL. It too can be ported from one database to another, or even across operating systems, with little or no modification. At least, that's the dream. In the real world, production-level code tends to require some tweaking in order to work in a new environment. This blog will outline some of the reasons that SQL syntax may differ across different database vendors.

The ANSI SQL Specification

ANSI, which stands for American National Standards Institute, defines the basic set of syntax rules and commands that are to be used to interact with relational databases. However, much like browser implementations of HTML, CSS, and ECMAScript, most database implementations of SQL are imperfect and/or incomplete. ANSI SQL allows for some flexibility in the level of conformance, so there is no strict vendor requirement to implement the full specification. But even at the basic, lowest level, all vendors diverge at least a little bit.

Beyond that, there are non-standard extensions, which all vendors support in one form or another. Even something as simple as indexes are non-standard. The ANSI SQL specification says nothing about indexes, so every vendor's implementation of indexing is a supplement to the standard. That opens the door for vendors to come up with whatever syntax they deem fit or most advantageous to their brand. The result: a variety of SQL dialects, which are largely the same, but with some distinctions.

Writing Versatile SQL

If you want SQL code that will work across all database types, you should stick to standard SQL statements like SELECT, WHERE, GROUP BY, ORDER BY, etc. Aggregate functions like SUM(), AVG(), MIN(), and MAX() will also be understood by all popular database types, including SQL Server, MySQL, PostgreSQL, SQLite, and Oracle. Here's a query that should work with any database:

Select        c.customer_id,    c.customer_name,    SUM(p.amount) AS total_salesFROM customers AS c    LEFT JOIN purchases AS p    ON c.customers_id = p.customer_idWHERE    c.customer_location = 'Canada'GROUP BY    c.customer_name ASC;

Learning SQL

If you're just starting out in database administration and/or development, you should concentrate on SQL that will apply to the most database types as possible. You should also work with a database that is highly ANSI SQL compliant and popular, such as MySQL. It has consistently been the most popular database for the past few dacades. It's also highly compliant, making it an excellent learning tool. There are many articles on it and most SQL samples were developed and run on MySQL. Microsoft SQL Server comes in at a close second. However, it uses Microsoft's dialect of SQL, called T-SQL. Having the most dissimilar SQL to other platforms makes SQL Server a less-than-ideal starter database. You're probably better off choosing PostgreSQL or SQLite, which are also quite popular and ANSI compliant. SQLite is particularly attractive to novices because of it's small size and portability.

Here are just some of the differences that you're likely to find between databases:

Case Sensitivity

Consider the clause WHERE name = 'Rob' Or WHERE name = 'rob':

Use of Quotation Marks

Some databases only support single quotes, while others allow both single and double quotes:

Column and Table Aliases

MySQL, PostgreSQL, and SQLite all use the "AS" keyword to denote aliases, i.e., SELECT SUM(score) AS avg_score, while SQL Server employs the equals sign, i.e., SELECT SUM(score) = avg_score.

Date/Time Functions

Each database implements its own date and time functions:

Navicat Premium: the Universal Tool

Navicat Premium is the tool of choice for working with a variety of database types. Not only can it connect to multiple databases simultaneously, but its Code Snippets feature makes writing queries against your preferred database type easier than ever before. The Code Snippets feature allows you to insert reusable code into your SQL statements when working in the SQL Editor. Besides gaining access to a collection of built-in snippets for common control flow statements and functions, you can also define your own.

You can download Navicat 17 for a 14-day fully functional FREE trial. It's available for Windows, macOS, and Linux operating systems.

It's a well established practice in database design and development to avoid storing any data that can be calculated or reconstructed from other fields. As a result, you may be missing some data when constructing your charts in Navicat BI. But that's not an issue, as Navicat BI provides Calculated Fields specifically for that purpose. In today's blog, we'll be using Calculated Fields to build a chart that shows the average rental times - i.e., how long a customer keeps a movie before returning it - per customer. As with most of the articles in this series, the data will be curated from the free "dvdrental" sample database.

Fetching the Customer Rental Information

As mentioned in previous blogs in this series, we should create the data source before designing the chart as we will need to specify the data source that the chart uses. Data sources reference tables in your connections or data in files/ODBC sources, and can select data from tables on different server types. After creating a new data source, we can click on "New Data Source Query" to open the Query Designer. There, we can write our SQL statement directly in the editor, use the visual Query Builder, or import a query from Navicat. Here's the SELECT statement that will fetch customer info, along with the rental amount, the date that the film was rented, and the date on which it was returned:

Once we save the query and refresh the data, we should see all the the query fields and result set:

We can now use the rental_date and return_date fields to calculate the rental duration. To do that, right-click the return_date in the field list (Control-click on macOS) and select New Calculated Field... from the context menu:

In the New Calculated Field dialog, you'll find all sorts of useful functions, including Aggregate functions, Datetime functions, Logic functions, and others. We'll use the DATEDIFF() function to calculate the number of days between the rental_date and return_date fields. The function accepts a time Unit, as well as a Start and End date. We can read the description below the function list for more information. We'll pass a "D" (day) for the unit, along with the two date fields, as follows:

There's a preview at the bottom of the dialog that tells us that we're getting the result we want.

After clicking the OK button, we should see our new field in the field list and results:

Building the Average Rental Duration Per Customer Chart

Since every customer ID is a separate data point, a scatter chart might work well. A scatter chart plots data with individual data points placed along the X and Y axes. We'll use the customer_id for the X axis and the rental_duration (Average) for the Y axis. Just drag the fields over to the X-Axis and Y-Axis fields in the chart designer, apply the Average aggregate to the rental_duration, and, presto, instant chart!

Here is the full chart in Present mode:

Bonus: Displaying the Number of Rentals Per Customer

While averages are helpful, it might also be useful to show how many times each customer rented one or more films. We can use an Aggregate function for this purpose. We'll count the number of amount entries in the result set and group them by customer_id. Here is the New Calculated Field dialog with the call to the AGGCOUNT() function:

In the Chart Designer, we'll drag our new number_of_rentals field to the Color slot. Adding an ascending sort will order the legend items from the lowest to highest number of rentals:

We can view the details of and individual data point by hovering the cursor over it. A tooltip will appear showing the number of rentals, the customer_id, as well as the average rental_duration in days:

Conclusion

This blog covered how to use Calculated Fields in your Navicat BI data sources and charts. These were just one of the new features included with the latest version of Business Insight (BI). This also brings us to the end of this series on Custom Fields. If you'd like to try Navicat BI, you can download the stand-alone version for a 14-day fully functional FREE trial. It's available for Windows, macOS, and Linux operating systems.

In Navicat BI, data sources reference tables in your connections or data in files/ODBC sources, and can select data from tables on different server types. The fields in the dataset can be used to construct a chart. In fact, when building a chart, you will need to specify the data source that's used to populate the chart.

As we've seen throughout this series, data sources support custom field types. These include: Type-Changed, Concatenated, Mapped, Custom-Sorted, and Calculated. The last blog covered how to use Custom-Sorted Fields to sort chart data according to a reference field. This week, we'll be learning how to set an explicit sort order. In order to do so, we will create a Vertical Bar Chart for the free "dvdrental" sample database that shows a sum of movie rental proceeds by month.

Configuring the Data Source

As mentioned earlier, our chart will require a data source that fetches the relevant data, so let's create a new data source named "Rentals by Month".

Here's a query that I created in Navicat for PostgreSQL:

We can now import it into our data source by clicking the Import Query button:

After refreshing the data, we can see the query fields and results:

Designing the Sales by Month Chart

Time to design our chart. First, let's see what happens when we sort by month name:

As you can see, this sorts the bars alphabetically according to the month name, and not in chronological order. To do that, we'll need to add a Custom-Sorted field to the data source by right-clicking the month (Control-click on macOS) in the field list and selecting New Custom Field -> New Custom-Sorted Field... from the context menu:

In the New Custom-Sorted Field dialog, we can now verify that the "Custom" radio button is selected, and proceed to move each month from the Suggested Values list into the Sorted Values using the arrow button (highlighted in red below):

If you ever make a mistake, no need to worry! You can just select the item and use the up and down arrows to change its position in the list.

Once you're satisfied with the sort order, click the OK button to close the dialog.

You should now see the new Custom-Sorted field in the query results:

Note that this will not affect the sort order in the data source, but it will once we add our new field to the chart and apply a sort to it.

If we now set the Custom-Sorted field as the chart Axis and sort it in ascending order, the bars will now follow the sort order that we assigned in the New Custom-Sorted Field dialog:

Conclusion

This blog covered how to use Custom-Sorted Fields to sort chart data according to an explicit sort order. Next week, we'll be moving on the final custom field type of the series: Calculated Fields.

You can download Navicat BI for a 14-day fully functional FREE trial. It's available for Windows, macOS, and Linux operating systems. You'll also find Navicat BI bundled with Navicat Premium and Enterprise Editions of Navicat for MySQL, Oracle, PostgreSQL, SQLite, SQL Server and MariaDB.

Navicat Premium has long been the choice of database professionals everywhere who needed to simultaneously connect to a variety of database platforms from a single application. Navicat Premium Lite now offers a streamlined database management experience for users who only require the core features needed for basic database operations. In today's blog, we'll go over all of the impressive features that you'll find in Navicat Premium Lite as well as where to download it for FREE.

The Main Window

In terms of viewing, updating, and deleting data, Navicat Premium Lite is virtually indistinguishable from its older brother, Navicat Premium. You can still create and modify records seamlessly in Grid View, Tree View and JSON View through the built-in editors. Only Form View is reserved for Enterprise Edition.

Object Designer

To help manage database objects such as tables, views, keys, and constraints, Navicat Premium Lite offers an Object Designer. It implements a clear and responsive interface that organizes the various objects into structured tabs. Here is the field list for a PostgreSQL table:

Tabs are configured according to the underlying database, so that connecting to PostgreSQL will cause the Rules tab to be present:

Query Editor

Thanks to Navicat Premium Lite's Query Editor, SQL coding has never been easier. It features code completion, code snippets, and syntax highlighting, all within an interface that is both clean and intuitive.

However, you'll need to move to the Enterprise Edition if you're looking to use the visual Query Builder of Beautify SQL tool.

Import and Export

You can import and export your data in a variety of text-based formats including TEXT (.txt), CSV, XML, and JSON. Only digital formats like Excel (.xls and .xlsx), MS Access and DBase are reserved for the Enterprise Edition.

Collaboration

You'll be happy to know that Navicat's collaboration tools such as Navicat Cloud and Navicat On-Prem Server are also available in Navicat Premium Lite. You will undoubtedly find Navicat Cloud to be particularly useful; it allows you to synchronize your connection settings, queries, snippets, and virtual group information to the cloud service so you can get real-time access to them, and share them with your coworkers, where ever they may be, all around the world.

Connect Securely

You can rest easy knowing that your connections are secured via SSH Tunneling and SSL. Moreover, advanced authentication methods, which include PAM, X.509, and GSSAPI, provide multiple layers of protection against unauthorized access.

Conclusion

Navicat Premium Lite is the perfect tool for users who desire all of the core functionality that's required for most database operations. If you take a look at the feature matrix, you'll see that all but the most advanced functionality is present in Navicat Premium Lite.

By retaining the ability to simultaneously connect to a variety of database platforms, including MySQL, Redis, PostgreSQL, SQL Server, Oracle, MariaDB, SQLite, and MongoDB, all from a single application, Navicat Premium Lite is sure to have an enormous impact on the free database management software landscape. Navicat Premium Lite is available for Windows, macOS, and Linux operating systems.

Welcome to the 3rd installment in this series on Creating Custom Fields In Navicat BI. In Part 1, we learned how to add Type-Changed Fields to your Navicat BI charts. Part 2 went on to describe how to use Concatenated Fields. Today's blog will introduce Mapped Fields. We'll be modifying the data source that we used in the last two articles, which connects to the free "dvdrental" sample database and returns a list of rentals for each film category. In the next blog, we will use the updated data source to create a chart that compares new releases to other categories.

Field Mapping Overview

In many ways, field mapping is highly similar to the process of transformation in Information Technology (IT). Whereas the latter runs a value through an algorithm to arrive at a transformed value, field mapping is simply the changing of one or more column values to another.

Field mapping can sometimes be observed in the field list clause of SELECT queries. For example, the "Sum of Payments per Movie Category" query which was the data source throughout this series returns a list of film categories along with a sum of their sales (or, more specifically, rentals). We can employ a CASE statement to make certain category names more descriptive, like say "Games" to "Video Games":

Creating the New Releases vs. Other Categories Data Source

Before designing any chart, we need a data source to fetch the information we require. Once you've got a few data sources, you might find it easier to repurpose an existing one rather than create a new data source from scratch. In fact, the Rentals by Category data source that we used last time will do nicely.

We can easily duplicate any item in the Navicat BI workspace by selecting and then right-clicking (or Control-click on macOS) it in the workspace, and selecting Duplicate <Item Type> from the context menu. Hence, the menu item that we would want is "Duplicate Data Source":

That will create a new data source named "Rentals by Category 1". To rename our new data source, click once on the item to select it and then a second time to activate edit mode. You can tell that the item is ready for editing when the label turns into a textbox with the item text highlighted in blue:

Let's call our new data source "New Releases vs. Other Categories". Press the Enter key to save the new name:

Adding a Mapped Field

To add a new Mapped Field to the data source, right-click the name field (or Control-click on macOS) and select New Mapped Field... from the context menu:

That opens the New Mapped Field dialog. There, let's begin by renaming the Target Field Name to "mapped_category_names".

Next, we'll map the "New" category name to something more descriptive. To do that:

• Since the "New" category will have a one-to-one mapping to the new value, select "One-to-One" from the Mapping Method drop-down.
• Choose "New" as the Source Value.
• Enter "New Release" for the Mapped Value.

Now we'll repeat the process for Null values, i.e., films which have not been assigned a category.

• Click the Add button and select "Add One-to-One Values..." from the context menu.
• In the Add One-to-One Values dialog, select the checkbox next to the (NULL) value and enter "Uncategorized" for the Mapped Value.

  

• Click the OK button to close the dialog and add the new row to the Mapped Fields table.

Finally, toggle the New Value radio button next to the Other Values label and enter "Other Categories" so that all other values are assigned to this catch-all category. The dialog should look as follows at this point:

Click OK to close the dialog. You should now see the mapped_category_names field in the data grid:

If you wish, you may delete the other calculated fields (as shown in the above image), since they won't be needed for the chart that we will be building next week.

Conclusion

This blog covered how to use Mapped Fields in your Navicat BI data sources. It is one of five custom field types, which include: Type-Changed, Concatenated, Mapped, Custom-Sorted, and, Calculated. Next week, we will use the "New Releases vs. Other Categories" data source to create a chart that compares new releases to other categories.

You can download Navicat BI for a 14-day fully functional FREE trial. It's available for Windows, macOS, and Linux operating systems.

On May 12th, Navicat added several major updates to existing products, including Navicat Premium, Navicat BI, and Navicat Data Modeler. One of the most popular Navicat tools, Navicat for MySQL, also benefitted from the new updates, receiving much of the same exciting new features as Navicat Premium. Today's blog will be covering just a few of the improvements that you'll find in the new Navicat for MySQL 17.

A Brand New Modeler!

We learned about Navicat Data Modeler 4 in a previous blog. It utilizes a single workspace that incorporates several databases, as well as models, diagrams, and other related objects. As we saw, this approach allows users to illustrate different model objects within a single diagram as well as facilitate efficient switching between models, cross-model management, and sharing of model workspaces. Navicat for MySQL 17 comes with the same functionality built-in.

Models support Functions and Procedures, which allows you to pre-define processes and operations during the modeling stage.

Data Dictionary

The Data Dictionary has become an integral document for both information systems and research projects. As the name suggest, the Data Dictionary contains names, definitions, and attributes about data elements stored within a database. We learned about the Data Dictionary as well as Navicat 17's new Data Dictionary tool in the Create a Data Dictionary in Navicat 17 blog article. Accessible from either the Model or main Navicat window, the Data Dictionary tool guides you though every step of the process of creating a highly professional finished document.

Powerful Data Profiling

In the May 7 blog article "Data Profiling in Navicat 17", we learned about the brand new Data Profiling tool. It provides a visual and comprehensive view of your data at the click of a button!You can find the Data Profiling tool in all Enterprise editions of Navicat database development and management tools, including Navicat for MySQL 17.

As you can see in the above screen capture, the Data Profiling tool offers a range of visual charts to represent the profiling results. These charts are a great way to analyze data types, formats, distributions, and informative statistics within your datasets.

Table Profiles

The "Exploring Table Profiles in Navicat 17" blog article explored the new Table Profile feature, which allows us to save different combinations of filters, sort orders, and column displays that are frequently used for the table.

Having the ability to save table profiles is a tremendous time saver because you can switch between multiple configurations quickly, without having to reconfigure the table each time you access it.

Improvements to the Query Designer

You'll be pleased to know that there have been a couple of notable improvements to the Navicat for MySQL Query Designer. These include:

• Pinned Results: query result may be retained for later reference by the click of a button!
• Visual Query Explain: the Query Explain feature has been enhanced to utilize graphics.

  

Manage Connections: Multiple Connection Properties In One Interface

Navicat for MySQL 17 offers the most straightforward process for initiating connections yet. It includes an advanced filter and search feature to help you quickly locate specific server types. You can also organize your connections with stars, colors and groups, or hide those which are seldom needed.

Other Notable Improvements

• Navicat URI: The server object URI may now be shared among team members, thus promoting collaboration.
• BI: As described in this Navicat BI tutorial, all charts on a dashboard using the same data source can now be interconnected. Hence, selecting any data point on one of the charts instantly updates all of the other charts on the same dashboard page that share the same data source to reflect your selection.

Conclusion

Today's blog covered just a few of the features that you'll find in the new Navicat for MySQL 17.

You can read more about all the new features and improvements in the Highlights page.

Navicat for MySQL 17 available for the Windows, Linux, and macOS operating systems on the product download page.

Welcome to part 2 in the Creating Custom Fields In Navicat BI series. Part 1 laid the groundwork for adding custom fields to your Navicat BI charts, starting with Type-Changed Fields. Today's blog will continue with Concatenated Fields. As with the last article, we'll be using a data source that connects to the free "dvdrental" sample database.

What Is Concatenation?

Concatenation is the appending of one string to another. It's commonly employed in queries to combine first and last name fields together. Case in point, the actor table in the "dvdrental" sample database splits actors' names into first_name and last_name fields. We can include both in a single column of the result by utilizing the concat() function:

Notice the passing of the space delimiter as the second input parameter; without it the names would be combined into a single word.

Adding a Concatenated Field To the Rentals by Category Data Source

In the recent blog on Type-Changed Fields we created a Vertical Stacked Bar Chart that shows daily sales for each movie category:

We'll now modify that chart so that categories include IDs, so that "Comedy" will now appear as "Comedy (5)".

To do that, we'll have to modify the "Rentals by Category" data source, which supplies the data that populates the chart.

Locate and double-click the "Rentals by Category" data source in the BI workspace (Hint: if you have a lot of items in your workspace, you can click on the "Data Source" toggle button to only show data sources):

To add a new Concatenated Field to the data source, select New Custom Field -> Concatenated Field... from the menu:

That opens the New Concatenated Field dialog. We can see that Navicat already included the category_id in the Body textarea. Place it within parentheses "()" and add the name field in front of it so that the contents of the Body field are:

["Sales per Category".name] (["Sales per Category".category_id])

In the Target Field Name, enter "category_id_and_name" and click OK to create the new field. Our new field will appear in the data grid with a blue header:

Updating Categories In the Average Sales by Date Chart

Now all that's left to do is replace the "name" field in the "Average Sales by Date" chart with our new Concatenated Field. To do that, you'll first need to open the chart by locating and double-clicking it in the BI workspace (Hint: if you have a lot of items in your workspace, you can click on the "Chart" toggle button to only show charts):

Next, we can simply drag-and-drop the "category_id_and_name" field from the data source field list to the chart Group:

That will instantly cause the chart to refresh. Notice that the legend values now include IDs:

Hovering the cursor over a vertical bar in the chart shows all of the data for that day, with the category under the cursor in bold:

Conclusion

This blog covered how to use Concatenated Fields in your Navicat BI data sources and charts. It is one of five custom field types, which include: Type-Changed, Concatenated, Mapped, Custom-Sorted, and, Calculated. Next week, we'll learn more about mapped fields.

You can download Navicat BI for a 14-day fully functional FREE trial. It's available for Windows, macOS, and Linux operating systems.

Back in the sneak peek at Navicat 17, we were introduced to a couple of new Business Intelligence (BI) features, namely Chart Interaction and Calculated Fields. It bears stating that Calculated Field are not the only type of custom field available in Navicat BI. In fact, there are five: Type-Changed, Concatenated, Mapped, Custom-Sorted, and, of course, Calculated. This blog will lay the groundwork for adding custom fields to your charts, starting with Type-Changed Fields. Over the next several weeks, each blog will cover a different field type. As in previous blog installments, we'll use a data source that connects to the free "dvdrental" sample database.

Changing a Field's Type

Here's the SELECT statement that fetches sales for each movie category:

SELECT  c.category_id,  c.name,  p.amount,  r.rental_dateFROM   payment as p    LEFT JOIN      rental AS r on p.rental_id = r.rental_id    LEFT JOIN      inventory AS i ON r.inventory_id = i.inventory_id    LEFT JOIN      film_category AS fc ON i.film_id = fc.film_id    LEFT JOIN      category AS c ON fc.category_id = c.category_idORDER BY c.category_id;

It is similar to the query that we saw in the previous tutorial on charts, but with two important differences:

• the field list includes the rental_date
• the query doesn't aggregate sales by category

We can see that the rental_date field contains a DateTime:

Now suppose that we'd like to remove the time portion of the dates. We could edit the underlying query, or, we could simply add a new Type-Changed field to the existing data source. To do that, we'll click on the rental_date header to select it and then click on the New Custom Field button and choose "Type-Changed Field..." from the context menu:

Having selected the rental_date column prior to clicking the New Custom Field button, Navicat knows to make a copy of that field. Let's call our new field "rental_date_no_time" and make it a Date type:

That will allow us to break down sales by date in charts.

After clicking the OK button, we can see the new field in the field list and data table:

Quick hint: If you ever need to convert a DateTime field into a timestamp, you can choose Number from the Target Type Field drop-down in the New Type-Changed Field dialog:

We can now use our new field in a chart. Here's a Vertical Stacked Bar Chart that shows daily sales for each movie category:

Customizing Dates In a Chart

It should be noted that we further customize the format of Date and Time fields in the chart itself. For instance, we could change the dates to a "DD MMM YYYY" format by selecting it from the Date Formats section of the Data properties:

The new format will be immediately reflected in the chart:

Conclusion

This blog covered how to use Type-Changed Fields in your Navicat BI data sources. It is one of five custom field types, which include: Type-Changed, Concatenated, Mapped, Custom-Sorted, and, Calculated. Over the next several weeks, we'll go over each of the remaining four custom field types.

You can download Navicat BI for a 14-day fully functional FREE trial. It's available for Windows, macOS, and Linux operating systems.

Business Intelligence (BI) is the practice of transforming data into actionable insights that empower organizations to streamline productivity and achieve better overall performance. This blog recently introduced the new Navicat BI, which is a tool that helps BI professionals better understand their data through the creation of data visualizations such as charts, graphs, and dashboards. Last week's blog walked us through the process of creating a workspace and data source in Navicat BI. Today's post will cover how to design an interactive chart and present it within a dashboard.

Building the Total Sales Percentages by Category Chart

Recall that, in last week's tutorial, we added the Dvdrental data source to our workspace. We will now use it to populate the Total Sales Percentages by Category chart.

To open the Chart Designer, click on the New Chart button in the main toolbar of the BI Workspace window:

That will bring up a dialog prompt where you can enter the Chart Name as well as assign the Data Source. Having previously created the Dvdrental data source, we can now select it to populate our new chart:

In the Chart Designer, the various chart types are located in a toolbar above the chart fields. We'll select The Pie Chart type by clicking its icon:

It has only two fields: the Group and Value. We can drag the fields from the data source on the left to the drop-downs. We'll group by name and show the value of the sum field:

Customizing the Chart

There are many ways to customize a chart via the Properties Pane to the right of the chart. For instance, we can set the title as follows:

There are many other properties in the Data pane. There, we can show or hide various data/chart elements as well as change the color palette. These may be selected from a predefined color scheme, or assigned to each specific value. Here's the chart without data values and a more colorful palette:

Presenting a Chart Within a Dashboard

Just as we did to create a new chart, we can click the New Dashboard button to create a dashboard:

Doing so will again present a dialog prompt, where we can assign a name to our dashboard. We'll call it "Total Sales Percentages by Category Dashboard":

A dashboard is a place to combine multiple views of data to glean richer insights. As such, a single dashboard may contain multiple (interconnected) charts, text, images, shapes, and other elements that help gain new insights on existing data. You can even give a dashboard a background image, as seen here in our film categories dashboard:

Dashboards may be saved as external files to be shared with colleagues or presented directly from Navicat BI.

Conclusion

This tutorial went through the process of creating a dashboard in Navicat BI. In part 1, we added a new workspace and built the data source. In today's blog, we learned how to design an interactive chart and present it within a dashboard.

You can download Navicat BI for a 14-day fully functional FREE trial. It's available for Windows, macOS, and Linux operating systems.

Navicat BI is a tool that helps you organize and transform your data into meaningful information through reporting. This is achieved via interactive dashboards that summarize the insights gained, along with workspaces that may be easily shared with colleagues and business leaders to make informed decisions on both operational (day-to-day) and strategic (long-term) matters. Today's blog will walk you through the process of creating a workspace and data source Navicat BI. Next week's post will cover how to design an interactive chart and present it within a dashboard.

The Chart at a Glance

The chart that we will be building will summarize how much was spent on each movie category of the free PostgreSQL "dvdrental" sample database. The data will be presented as a pie chart where each piece will represent a category. Here's a sneak peek at what the chart will look like:

Creating the Workspace

Navicat BI is available as a stand-alone product and is also integrated into Navicat Premium and Enterprise Editions. For the purposes of this tutorial, we'll be working in Navicat Premium 17.

The first step is to create a new workspace. To do that:

• Locate and click the BI button in the main Button Bar at the top of the main Navicat window.
• Click the New Workspace button:

  

That will launch the BI feature:

Creating the Data Source

The BI feature lets you specify and integrate data from a variety of data sources, including databases (or any ODBC data source), external files such as Excel, Access, CSV, and even data stored on your computer, network, or a URL.

The new workspace clearly shows the steps to create a data visualization:

• Create data source
• Design chart
• Present your dashboard

Since I already have the dvdrental database in Navicat, I'll build the query there and then import it into the BI workspace.

Here is the full SQL statement. It includes an aggregation on the amount column of the payment table that sums its values for each category:

SELECT  c.category_id,  c.name,  sum(p.amount)FROM   payment as p    LEFT JOIN      rental AS r on p.rental_id = r.rental_id    LEFT JOIN      inventory AS i ON r.inventory_id = i.inventory_id    LEFT JOIN      film_category AS fc ON i.film_id = fc.film_id    LEFT JOIN      category AS c ON fc.category_id = c.category_idGROUP BY c.category_id, c.nameORDER BY c.category_id;

Here is the above query in the Navicat Query Editor, along with the results. Note that the query was saved with the name "Sum of Payments per Movie Category". We'll need to recall the name later in order to import the query:

Now we'll create the data source in BI workspace:

• Click the New Data Source button at the top of the BI workspace window.
• Name the data source "Dvdrental" and select PostgreSQL for the Database connection:

  

• Click Next to continue.
• Select the PostgreSQL connection that contains the dvdrental database (I only have the one) and click OK to create the data source:

  

We can now see the PostgreSQL connection in the Connections pane. If we expand it to see the dvdrental database, we can see the New Data Source Query item above the tables. Clicking it will open a new query editor. We could write the query there, but since we already did, we can click the Import Query button instead:

That will launch the Import Query dialog, where we can select the query that we built earlier:

Click the Import button to add it to our workspace.

Going Forward

With the data source in place, we're ready to design the chart. We'll do that in next week's blog. In the meantime, feel free to familiarize yourself with Navicat BI's many chart types, which include Bar Charts, Line/Area Charts, Bar/line Charts, Pie charts, Heatmap/Treemap, Pivot Table, Waterfall Chart, Scatter Chart, Value, Control, KPI/Gauge, and more!

You can download Navicat BI for a 14-day fully functional FREE trial. It's available for Windows, macOS, and Linux operating systems.

Business Intelligence (BI) is the practice of transforming data into actionable insights that empower business leaders to enhance overall performance. Once of the most important phases of this process is the data exploration and visualization phase. It entails the organization and transformation of data into meaningful information through reporting. To make data more understandable, BI professionals create data visualizations such as charts, graphs, and dashboards. These visual representations help decision-makers quickly grasp complex information. That's where Navicat BI comes in. Formerly known as Navicat Chart Creator, Navicat BI produces interactive dashboards that summarize the insights gained. Workspaces may be easily shared with colleagues and business leaders to make informed decisions on both operational (day-to-day) and strategic (long-term) matters. Today's blog article will touch upon a few of Navicat BI's many features.

Choose from a Variety of Data Sources

Navicat BI lets you specify and integrate data from a variety of data sources with ease. Not only does it come with pre-built data connectors for databases such as MySQL, PostgreSQL, SQL Server, Oracle, SQLite, MariaDB, MongoDB and Snowflake, but it can also import data from any ODBC data sources including Sybase and DB2.

Data sources are not limited to databases either; data may reside within external files such as Excel, Access, CSV, as well as from data stored on your computer, network, or a URL.

Regardless of data source type(s), charts are updated in real time so that they always reflect any changes in the underlying data.

Custom Charts

To better inspire your organization to make better decisions, Navicat BI provides a wealth of different chart types to best illustrate your data in a meaningful way. These include Bar Charts, Line/Area Charts, Bar/line Charts, Pie charts, Heatmap/Treemap, Pivot Table, Waterfall Chart, Scatter Chart, Value, Control, KPI/Gauge, and more. By selecting the best chart type for your data ensures that your questions are easily answered and your presentations convey the message you wish to communicate.

Effective Dashboards

Dashboards provide an interactive display of your charts. A single dashboard may combine multiple views of data to glean richer insights. All of the charts on a dashboard that share the same data source may be interconnected so that selecting a data point on one of the charts instantly updates all the other charts to reflect your selection.

Collaboration Made Easy!

Navicat BI is fully integrated with Navicat Collaboration, allowing you to synchronize your BI workspaces to Navicat's cloud solutions. You can then invite your teammates to the project, where they can create and edit together.

Conclusion

This blog article covered just a few of Navicat BI's many features. In future articles, we'll delve into how to use Navicat BI in greater detail.

Navicat BI is available as a stand-alone product and is also integrated into Navicat Premium and Enterprise Editions. You can download it for a 14-day fully functional FREE trial. It's available for Windows, macOS, and Linux operating systems.

Having covered Navicat version 17 over the course of the last few weeks, it's time that we turned our attention to two other noteworthy releases, namely Navicat Data Modeler and Navicat BI (previously called Navicat Chart Creator). Today's blog will introduce Navicat Data Modeler 4, while Navicat BI will be featured next week.

Navicat Data Modeler 4: First Glance

Navicat Data Modeler is a stand-alone product that features many powerful capabilities, without sacrificing and user-friendliness. It's perfectly suited to data modeling tasks of any complexity, making it an ideal choice for users of all levels, from novice to professional. Even those who are new to modeling will find Navicat Modeler 4's intuitive interface and seamless experience to be of great value.

Craft All of Your Models In a Unified Space

In Navicat Modeler 4, a single workspace may incorporate several databases of different types, along with models, diagrams, and other related objects. This approach allows users to visualize different objects within the same diagram as well as facilitate efficient switching between models, cross-model management, and sharing of model workspaces. The end result is increased collaboration, which leads to enhanced overall productivity. Finally, the use of workspaces simplifies the navigation of complex systems, as well as promote a better understanding of system components.

Design Your Diagrams with Ease

Navicat Modeler 4's environment is both responsive and interactive, making diagram creation amazingly easy. You're not constrained to employing a dogmatic diagramming style or approach. By contrast, Navicat Modeler 4 supports many different types of models, notations, and representations. By keeping everything simple and concise, you are better able to concentrate on designing your models. Some notable features include:

• Layers: related elements may be arranged together on separate layers.
• Locking/grouping option: elements may also be grouped together, so that they stay in place or move together as a single unit during editing or repositioning.

  

• Auto-layout upgrade: as the name suggests, the auto-layout command automatically repositions and aligns elements in a visually pleasing way. Auto-layout may be applied to the entire diagram, selected elements, or all elements under the same layer.
• New Present Mode: displays the model in a full-screen view, removes distractions, and provides a focused view of the diagram.

Data Dictionary

The new Data Dictionary provides documentation and descriptions for each data element within databases across various server platforms. A wizard guides you though every step of the process to create a highly professional finished document. Documents may be exported as PDFs for sharing with team members and other stakeholders.

Keep your Model and Database Synchronized

Over time, models can become out-of-date as the database evolves. That's where Navicat Modeler 4's Synchronization tool can help. By comparing and updating your model based on changes made in the database, you can always be certain that the model accurately reflects the current database structure. Synchronizing regularly minimizes discrepancies between the model and the database, thereby maintaining model integrity.

Compare Modeling Projects

The Compare Model Workspace helps find and highlight all the differences between both internal and external workspaces in mere minutes. Maintaining consistency across different versions or branches of the model ultimately leads to superior models all through the development and deployment process.

Conclusion

Today's blog gave us a preview of what we can expect from Navicat Data Modeler 4. You can download it for a 14-day fully functional FREE trial. It's available for Windows, macOS, and Linux operating systems.

Navicat 17 gives you more ways to connect to your database instances than ever before. In addition to traditional connection strings, Navicat 17 also supports URI connections. These provide access objects with ease, regardless of their location. There's also a new Manage Connection feature, which allows you to establish and manage connections through a single user-centric interface. In today's blog, we'll learn about both these exciting new features and how to use them to organize our connections with stars, colors and groups.

How URIs Provide Direct Access for Seamless Collaboration

Having the ability to share the server object URI among team members is a surefire way to increase collaboration. Navicat now offers a convenient shortcut to access the server object seamlessly, regardless of team members' locations. Simply clicking on the URI immediately opens the object in Navicat, thus avoiding having to manually locate the object. Saving time in in this way enables team members to concentrate on their tasks without burdening them with additional work.

Establishing a Connection

Thanks to Navicat's user-centric interface, it's never been easier to connect to both local and cloud database instances. It greatly simplifies the connection setup, catering to users with varying levels of technical knowledge. The advanced filtering and search capabilities enable rapid and precise location of specific server types. The ability to manage multiple connection profiles and create URI-based connections further enhances efficiency as well as the overall user experience.

Advanced Connection Management Features

The Manage Connections feature introduces a new method for handling multiple connection properties from a central location, enabling efficient batch operations. You can tailor your connection management to your specific needs by prioritizing key connections with stars, assigning colors based on their importance, or grouping them together. With Manage Connections, everything is neatly organized and easily accessible, saving time and effort when searching for specific connections.

To launch Manage Connections, hover the cursor over the "My Connections" header in the Object Explorer pane. You'll see a cog wheel appear on the right-hand side.

Click it to open the Manage Connections tab:

Grouping Connections

You may wish to group connections together by project, server, or any number of shared attributes. To create a new group in Navicat, simply right-click (or Control-click in mac) anywhere in the Manage Connections pane and select Manage Group -> New Group from the context menu:

A new folder will then appear with a textbox beside it, ready to accept the group name.

Once a group folder has been created, you can drag any connections into it to group connections.

Adding a Color

Similar to grouping, assigning colors to connections can help organize them. To assign a color to a connection, right-click (or Control-click in mac) the connection name and select a Color from the context menu:

Assigning a Star

Stars are another way to highlight particular connections. To add a star to a connection, right-click (or Control-click in mac) the connection name and choose the "Add Star" command from the context menu:

Conclusion

In today's blog, we explored the many ways that Navicat 17's single user-centric interface allows us to establish and manage connections more efficiently than ever before.

Now that Navicat 17 is out, you can download it for a 14-day fully functional FREE trial. It's available for Windows, macOS, and Linux operating systems.

With less than a week to go before the release of Navicat 17, this is the perfect time to delve into the new Table Profile feature. It allows us to save different combinations of filters, sort orders, and column displays that are frequently used for the table. So, without any further ado, let's get started!

Creating a Table Profile

The free "dvdrental" sample database contains a number of tables, views, and functions pertaining to the running of a fictional movie rental store. Here's the rental table in Navicat 17:

By default, tables are shown in the Grid Viewer, with records sorted according to the table design, or by whatever order they were added to the table. Columns are presented in the same order in which they appear in the table design, from left to right.

However, we can change a table's appearance in several ways. For instance, we can:

• apply filtering and sorting
• freeze certain columns in place
• hide columns
• change the column order
• alter a column's width

Let's try a couple actions on the rental table.

To change a column's position, we can drag it to wherever we want it. Likewise, we can change the width of a column by dragging its edge.

Here is the rental_date column after being dragged to the first position and resized to be a little more compact:

We can also quickly sort a column by clicking the arrow in the top-right corner of the column header and selecting an option from the context menu:

Notice that the arrow now shows the sort order:

Saving a Table Profile

If we were now to close the rental table, the next time that we access it, all of the changes that we made would be lost. However, in Navicat 17, we can save them to a new Table Profile. To do that, we would click the Table Profile button in the Table Toolbar and then select either the "Save Profile As" menu command ("Save Profile" would also work, as we haven't saved a profile for the table yet):

A dialog prompt will appear where we can choose a name for our profile. After providing a descriptive name, we can click OK to create the new Table Profile:

Loading a Table Profile

As mentioned previously, reopening the rental table will cause our changes to be lost. Of course, having created a Table Profile, we can restore the table to its previous state by accessing it via Table Profile -> Load Profile -> [profile name]:

Going Back to the Default Profile

If ever you want to flip back to the default profile, you can do that by selecting Load Profile -> Default Profiles -> Quick Mode:

Managing Profiles

Selecting Table Profile -> Manage Profile from the Toolbar brings up the Profile screen. There we can see which columns have been hidden, as well as what filters and sorting have been applied. It even shows a preview of the SQL utilized to fetch the underlying data:

We can delete or load any profile by selecting it and clicking on the Delete or Load button.

One final tip: we can open the directory on our device that contains the Profile definition files by right-clicking a profile (or Control-Click on mac) and choosing the Open Containing Folder... command from the context menu:

Conclusion

In today's blog, we learned how to work with Navicat 17's new Table Profiles by creating one for the "dvdrental" sample database's rental table.

On May 13, be sure to visit the Navicat Premium (English Edition) product page to learn more about version 17!

Navicat 17, which is due for release on May 13 (for English Edition), adds many new and exciting features. One of these is the Data Dictionary tool. It employs a series of GUI screens to guide you through the process of creating a professional quality document that provides descriptions for each data element within databases across various server platforms. In today's blog, we'll learn more about Data Dictionaries, as well as go through the steps to create one in Navicat 17.

What Is a Data Dictionary?

A Data Dictionary contains names, definitions, and attributes about data elements stored within a database. It can also be used for data that is defined as part of an information system or research project. It describes the meaning and purpose of each data element, and offers guidance on their interpretation, accepted meanings and representation. Additionally, a Data Dictionary provides metadata concerning these data elements, which helps in delineating their scope, characteristics, as well as the rules which govern their usage and application.

Data Dictionaries are beneficial for several reasons. They:

• Ensure consistency and avoid data discrepancies throughout a project.
• Establish standardized conventions for project-wide use.
• Foster consistency in the collection and use of data among team members.
• Facilitate streamlined data analysis.
• Promote adherence to data standards.

Creating a Data Dictionary for the PostgreSQL dvdrental Database

The PostgreSQL "dvdrental" sample database is a free download that you can use for learning and practicing PostgreSQL. As you might have guessed from its name, the DVD rental database represents the business processes of a DVD rental store.

We'll use Navicat 17's Data Dictionary tool to create documentation for it.

Selecting the Database(s)

Launching the Data Dictionary tool is easy; just select Tools -> Data Dictionary... from the main menu:

That will launch the first in a series of dialogs that will walk us through the process of creating the Data Dictionary. A process such as this one that guides users step-by-step through a series of tasks is known as a wizard.

This first dialog lets us select the database(s) that we'd like to document. Note that, if we select the database in the Navigation Pane before launching the Data Dictionary wizard, it will be pre-selected in the dialog.

Click the Next button to continue.

Choosing Objects

On the next screen, we can choose the database objects that we'd like to include in our Data Dictionary, as well as reorder the server, databases, and schemas.

Be default, the tool will generate some high level information about the database as well as definitions for all tables, views, and functions. However, we can choose to omit any of these if we wish by deselecting the associated checkbox.

There is a Search bar at the bottom of the screen to help locate objects:

Templates

Navicat provides a number of fully customizable templates:

Once we've selected a template, we can go ahead and customize every facet of its appearance.

Different parts of the document are accessible via the headers at the top of the screen. These include the:

• Cover
• Table of Content
• Header/Footer
• Main Content
• Paper

Generating the Document

The last step is to save the Data Dictionary to a file. The document will be saved as a Portable Document Format (PDF) file.

Every step of document creation will be logged in real-time so that we can see if Navicat encountered any issues along the way.

We can click the Open button at the bottom of the screen to view the final product in the associated program:

Conclusion

In today's blog, we learned more about Data Dictionaries, as well as how easy it is to create one in Navicat 17. You can learn more about version 17 by visiting the Navicat Premium (English Edition) product page from May 13 onward!

Last week's blog heralded the upcoming launch of Navicat 17, which is currently in Beta and scheduled to arrive on May 13 (English Edition)! As we saw, version 17 introduces a lot of exciting new features. One of the biggest is the Data Profiling tool. It provides a visual and comprehensive view of your data at the click of a button! In today's blog, we'll use it to obtain some quick statistics on the rental table of the free PostgreSQL "dvdrental" sample database.

Launching the Data Profiling Tool

As mentioned in the introduction, the Data Profiling tool requires little more than the click of a button to use. You'll find it in the toolbar of any table, view, or query result (highlighted in red below):

From there, you can choose to profile all records (the default) or add a filter to only profile rows which fit a given criteria:

Filtering Records

For datasets with many records, it is often useful to focus on a subset of the data. That's where the "Add Filter" option comes in. It allows us to add filters (and sorting) using the familiar "Filter & Sort" feature. Let's say that we only want to profile records of the rental table whose rental date is in the first half of 2006. All we need to do is add a filter on the rental_date column that selects rows with values that are between Jan 1st, 2006 00:00:00AM and June 30th, 2006, at 11:59:59PM. Selecting the dates and times is a snap, thanks to the built-in Date and Time Picker!

One feature of the Data Profiler that you won't find in the "Filter & Sort" tool is the ability to limit records to a certain number, like say, a thousand:

Viewing Profiling Results

Clicking the "Start Profiling", or ""Apply Data Settings" button after editing the criteria, runs the profiler on the rows which fit the selected filtering criteria.

Clicking on the column header shows the statistics for that field. These are shown in 2 places: under the column name and below the grid.

The kinds of stats you'll find include the percentage of Nulls vs. Non-nulls, as well as the number of distinct and unique values. There's even a value distribution chart! To view all of the values, you can either increase the column width or simply use the scrollbar at the bottom of the Value Distribution chart in the Column Statistics at the bottom of the screen:

Changing the Layout

There are a few options for changing how the data is presented. For instance, we can show distributions by count or value:

We can also choose between a Compact or Detailed layout (Detailed is the default). Here are the rental table headers with the Compact layout:

Getting More Specific

Every bar of the distribution chart represents a real record in the underlying table, view, or query. We can learn more about it by hovering the cursor over it. The popup box shows the value, along with how many times it appears within the dataset and what percentage that represents across all of the records:

Moreover, clicking a bar will Spotlight that record, which will home in on that row in the grid and display statistics which are pertinent to that value:

Clicking the bar a second time will remove the Spotlight.

We can also see in the above image the full range of stats available in the Column Statistics section. It includes additional figures, such as the number of Repeated values, Minimum and Maximum values, and many more.

Conclusion

In today's blog, we familiarized ourselves with Navicat 17's new Data Profiling tool by using it to obtain some quick statistics on the free "dvdrental" sample database's rental table.

On May 13, be sure to visit the Navicat Premium (English Edition) product page to learn more about version 17!

One of PostgreSQL's key features is the ability to enforce various constraints on data, ensuring data integrity and reliability. Today's blog article will provide an overview of PostgreSQL's various constraint types and explore their usage with examples from the free "dvdrental" sample database.

1. Check Constraints:

Check constraints allow you to specify conditions that must be met for a column when inserting or updating data. This ensures that only valid data is stored in the database. For instance, if you have a "customers" table and want to ensure that the age of a customer is at least 18, you can add a check constraint like this:

ALTER TABLE customersADD CONSTRAINT check_age CHECK (age >= 18);    

2. Not-Null Constraints:

Not-null constraints ensure that a column cannot contain null values. For example, in the "customers" table, if you want to ensure that every customer has a valid email address, you can enforce a not-null constraint on the email column like this:

ALTER TABLE customersALTER COLUMN email SET NOT NULL;    

3. Unique Constraints:

Unique constraints ensure that the values in a column or a group of columns are unique across all the rows in a table. This is often used for fields like usernames or email addresses to avoid duplication. For instance, in the "customers" table, if you want to ensure that each customer has a unique email address, you can add a unique constraint like this:

ALTER TABLE customersADD CONSTRAINT unique_email UNIQUE (email);    

4. Primary Keys:

A primary key is a combination of unique and not-null constraints. It uniquely identifies each record in a table and ensures data integrity. In the "customers" table, you might have a column named "customer_id" that serves as a primary key:

ALTER TABLE customersADD CONSTRAINT pk_customer_id PRIMARY KEY (customer_id);    

5. Foreign Keys:

Foreign keys establish a relationship between two tables by enforcing referential integrity. They ensure that values in one table's column match values in another table's column. For example, in the "rental" table, if you want to ensure that every rental record references a valid customer, you can add a foreign key constraint like this:

ALTER TABLE rentalADD CONSTRAINT fk_customer_idFOREIGN KEY (customer_id)REFERENCES customers(customer_id);    

6. Exclusion Constraints:

Exclusion constraints ensure that no two rows in a table satisfy a specified predicate. This allows you to define custom constraints beyond simple unique or check constraints. For example, you might have a "bookings" table where you want to ensure that no two bookings for the same room overlap in time:

ALTER TABLE bookingsADD CONSTRAINT exclude_overlapping_bookingsEXCLUDE USING GIST (room_id WITH =, booking_range WITH &&);    

Constraints in Navicat

Navicat for PostgreSQL 16 offers an easy-to-use graphical Table Designer for creating and managing PostgreSQL constraints:

Primary Key constraints are created when you add a key icon to one or more fields by clicking in the Key column. Other constraints are found on their associated tab.

Conclusion

PostgreSQL provides several different types of constraints to maintain data integrity and enforce business rules. Understanding these constraints and how to use them effectively is essential for designing robust and reliable database schemas.

Looking for an easy-to-use graphical tool for PostgreSQL database development? Navicat 16 For PostgreSQL has got you covered. Click here to download the fully functioning application for a free 14 day trial!

It's official: Navicat 17 (English Edition) is currently in Beta and scheduled to launch on May 13! If you thought that Navicat 16 introduced a lot of exciting new features, you may want to sit down for this. There is so much ground to cover for this upgrade that we can barely get through all of them in one article. Nonetheless, today's blog will at least provide an outline of what to expect come May 13 (for English Edition).

All-New Model Workspace

The Model Workspace has been completely redesigned to include enhanced diagram design, a more powerfulsynchronization tool, support of data dictionary, and more.

Data Profiling

The Data Viewer now integrates a Data Profiling tool that provides a visual and comprehensive view of your data.

Data Dictionary

The new Data Dictionary provides documentation and descriptions for each data element within databases across various server platforms. A wizard guides you though every step of the process to create a highly professional finished document:

Query Pinned Result

Clicking the Pin button on any query result retains it for later reference.

Query results may be just as easily discarded using the Unpin button.

Visual Query Explain

Available in MySQL, MariaDB and PostgreSQL, Visual Query Explain can help you gain valuable insights on query implementation in ways that the traditional text Explain simply can't.

Table Profile

Now you can save different combinations of filters, sort order, and column displays that are frequently used for the table.

You can also see in the above screenshot that you now have the option to show the data types in the column headers.

Navicat URI

This feature allows team members to share and locate server objects with ease.

Manage Connection

Navicat 17 helps you to organize your connections with stars, colors and groups, or even hide them.

Business Intelligence (BI) Feature

The Business Intelligence (BI) includes a couple of additions:

• Chart Interaction: All charts on a dashboard can be interconnected.
• Calculated Field: Data can now be transformed using specific formulas or expressions.

Visual Aggregation Pipeline

You can now construct and test your MongoDB aggregation pipelines step-by-step through a clear and responsive UI.

Support for Redis Sentinel

Navicat has long supported Redis, the popular open-source in-memory data structure store. Navicat 17 adds support for Redis Sentinel. The high availability solution for Redis, Redis Sentinel provides monitoring, automatic failover, and configuration management for Redis instances, ensuring continuous operation even in the event of failures.

Conclusion

This blog covered just some of the exciting new features to expect in Navicat 17 on May 13 (for English Edition). We'll be sure to cover each of them in more detail in future blog instalments!

PostgreSQL, the popular open-source relational database management system, offers various index types to optimize query performance and enhance data retrieval efficiency. In this article, we'll learn how to create different types of indexes in PostgreSQL. Wherever possible, indexes will be applied to the free "dvdrental" sample database using both DML statements as well as Navicat for PostgreSQL 16.

1. B-Tree Index:

The B-Tree index is the default index type in PostgreSQL, suitable for various data types, including text, numeric, and timestamp. It organizes data in a balanced tree structure, facilitating efficient range queries and equality searches. Let's create a B-Tree index on the "customer_id" column in the "payment" table:

CREATE INDEX btree_customer_id_idx ON payment(customer_id);

In Navicat you'll find indexes on the "Indexes" tab of the Table Designer. To create the above index, we would enter "btree_customer_id_idx" in the Name field, choose "customer_id" for the "Fields", and select "B-Tree" from the Index method drop-down:

Here is the btree_customer_id_idx index with all of the above fields populated:

Clicking the Save button with then create the index.

2. Hash Index:

Hash indexes are optimal for equality checks but less effective for range queries. They use hash functions to map keys to index entries. Here's how to create a Hash index on the "film_id" column of the "inventory" table, first using a DML statement:

CREATE INDEX hash_film_id_idx ON inventory USING HASH(film_id);

And now with Navicat:

3. GiST Index:

Generalized Search Tree (GiST) indexes support various data types and complex queries, making them versatile for applications like full-text search and geometric data types.

Here's an example of creating a GiST index on a geometry column:

CREATE INDEX index_geometry ON table_name USING GIST (geometry_column);

4. SP-GiST Index:

Space-Partitioned Generalized Search Tree (SP-GiST) indexes are suitable for data types with multidimensional or hierarchical structures. They efficiently index non-balanced trees.

Here's an example of creating an SP-GiST index on a tsvector column:

CREATE INDEX index_text_search ON table_name USING SPGIST (tsvector_column);

5. GIN Index:

Generalized Inverted Index (GIN) is ideal for cases like full-text search, array types, and composite data types. They are efficient for data types with multiple keys or components. Let's create a GIN index on the "title" column in the "film" table for full-text search:

CREATE INDEX gin_title_idx ON film USING gin(to_tsvector('english', title));

Here is the Indexes tab of the "film" table in Navicat with the gin_title_idx index added:

6. BRIN Index:

Block Range Index (BRIN) is suitable for large tables with sorted data, as it indexes ranges of data blocks rather than individual rows. It is efficient for columns with correlation between adjacent values. Here's how to create a BRIN index on the "rental_date" column in the "rental" table:

CREATE INDEX brin_rental_date_idx ON rental USING brin(rental_date);

Here is the brin_rental_date_idx index in Navicat:

Conclusion

PostgreSQL offers a range of index types catering to diverse data types and query requirements. Understanding the characteristics of each index type helps database administrators and developers to make informed decisions when optimizing database performance. Meanwhile, using a tool like Navicat for PostgreSQL 16 makes working with indexes much easier.

PostgreSQL rules offer a powerful mechanism for controlling query execution and enforcing data manipulation within the database. Understanding the syntax and usage of rules is essential for harnessing their capabilities effectively. In last week's article, we explored how PostgreSQL rules work and how they differ from triggers. Today's follow-up will cover their syntax in detail with more practical examples using the free "dvdrental" sample database.

Anatomy of PostgreSQL Rules

PostgreSQL rules consist of several key components that define their behavior:

• CREATE RULE Statement: To create a rule, we use the CREATE RULE statement followed by a rule name and the rule definition.
• Rule Event: Rules can be triggered by various events, including SELECT, INSERT, UPDATE, DELETE, or a combination (ALL).
• Rule Action: The action specifies what should happen when the rule is triggered. It can be an SQL statement such as SELECT, INSERT, UPDATE, DELETE, or a custom action.
• Rule Condition: Conditions are optional and allow rules to be triggered only when certain criteria are met. They are specified using a WHERE clause.

Practical Examples Using the "dvdrental" Sample Database

Example 1: Auditing Inserts

Suppose we want to log all insertions into the "customer" table for auditing purposes. First we'll need a table to store the audit data:

CREATE TABLE customer_audit (    action_type VARCHAR(10),    customer_id INT,    audit_timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP);

We can also create the above table using Navicat for PostgreSQL 16's Table Designer. Here's what that looks like:

Now we'll create a rule that inserts a record into an audit table whenever a new customer is added:

CREATE RULE log_customer_insert AS    ON INSERT TO customer    DO ALSO        INSERT INTO customer_audit (action_type, customer_id)        VALUES ('INSERT', NEW.customer_id);

In Navicat, you'll find the rules for a given table on the "Rules" tab of the Table Designer. Here is the log_customer_insert rule:

Example 2: Restricting Updates

Let's say we want to prevent updates to the rental return date once it has been set. We can create a rule that blocks any attempts to update the return date column after it has been initially set:

CREATE RULE prevent_return_date_update AS    ON UPDATE TO rental    WHERE OLD.return_date IS NOT NULL AND NEW.return_date IS DISTINCT FROM OLD.return_date    DO INSTEAD NOTHING;

Here is the prevent_return_date_update rule in Navicat:

You may recognize the enforce_min_rental_duration rule from last week's article.

Example 3: Data Transformation

Suppose we want to transform the format of phone numbers stored in the "address" table from international format to local format. We can create a rule that automatically updates phone numbers whenever a new address is inserted:

CREATE RULE transform_phone_number AS    ON INSERT TO address    DO ALSO        UPDATE address        SET phone = '+1-' || SUBSTRING(phone FROM 3)        WHERE address_id = NEW.address_id;

Need more space to enter the complete Where or Definition statement? Clicking the ellipsis [...] button beside the text box opens a large text area where you can view and compose the full statement. Here is the transform_phone_number rule in Navicat that shows the full Definition:

Conclusion

PostgreSQL rules offer a versatile toolset for implementing complex logic and enforcing data integrity within the database. By exploring diverse examples like auditing inserts, restricting updates, and data transformation, developers can gain a deeper understanding of how rules can be applied to address various requirements effectively. With PostgreSQL's flexible rule system, developers can tailor database behavior to meet specific business needs while ensuring data consistency and reliability.

PostgreSQL, a powerful open-source relational database management system, offers various features to enhance data management and manipulation. Among these features are rules, a mechanism used to control how queries and commands are processed within the database. In this article, we will explore how PostgreSQL rules work and how they differ from triggers, with a practical example using the free DVD Rental Database.

What are PostgreSQL Rules?

PostgreSQL rules provide a way to rewrite queries or commands before they are executed. They act as a set of predefined actions to be performed automatically based on certain conditions. Rules are primarily used to implement data abstraction and customization without altering the underlying schema.

Furthermore, PostgreSQL rules offer a powerful mechanism for enforcing business logic within the database itself, reducing the need for application-level constraints and ensuring consistent data manipulation across different applications or interfaces. By encapsulating complex logic within the database, rules promote data integrity and maintainability while simplifying the development process.

How do Rules Differ from Triggers?

While rules and triggers serve similar purposes in PostgreSQL, there are notable differences between the two.

1. Execution Time:
   • Rules: Rules are applied during query parsing, meaning they affect the query plan generation.
   • Triggers: Triggers are executed after the completion of an event such as INSERT, UPDATE, or DELETE.
   
   
2. Visibility:
   • Rules: Rules are transparent to users executing queries. The rewritten query is visible in the query execution plan.
   • Triggers: Triggers are explicitly defined on tables and are triggered by specific events.
   
   
3. Granularity:
   • Rules: Rules can be applied at the table level or view level, providing more flexibility in customization.
   • Triggers: Triggers are bound to specific tables and cannot be applied globally.
   
   
4. Complexity:
   • Rules: Rules can be complex and may involve multiple actions or conditions.
   • Triggers: Triggers are simpler to implement and manage as they are event-driven.

Practical Example Using the "dvdrental" Sample Database: Enforcing Data Validation

Let's explore a practical example to understand how PostgreSQL rules work in conjunction with the "dvdrental" sample database.

Suppose we want to enforce a constraint where rental durations must be at least one day. We can achieve this using a rule:

CREATE RULE enforce_min_rental_duration AS    ON INSERT TO rental    WHERE (NEW.return_date - NEW.rental_date) < INTERVAL '1 day'    DO INSTEAD NOTHING;

In Navicat we can add a rule in the "Rules" tab of the Table Designer. The "Do instead" drop-down lets us choose between "INSTEAD" and "ALSO". Meanwhile, the "Where" textbox accepts the criteria for executing the rule and the "Definition" box describes what the rule should do. Here is the complete rule definition in Navicat:

This rule ensures that any attempt to insert a rental with a duration less than one day is prevented.

Conclusion

PostgreSQL rules are a powerful tool for controlling query execution and enforcing data integrity. While similar to triggers, they offer distinct advantages in terms of execution time, visibility, granularity, and complexity. By understanding the differences between rules and triggers and leveraging their capabilities, developers can effectively customize database behavior to meet specific requirements while maintaining data integrity and security.

Interested in giving Navicat 16 For PostgreSQL a try? You can download the fully functioning application here to get a free 14 day trial!

Data integrity is a critical aspect of any database system, ensuring that the data stored remains accurate, consistent, and meaningful. In PostgreSQL, one powerful tool for maintaining data integrity is the use of check constraints. These constraints allow you to define rules that data must adhere to, preventing the insertion or modification of invalid data. In this article, we'll explore how to use check constraints to validate data in PostgreSQL, using the free DVD Rental Database as a reference.

Understanding Check Constraints

Check constraints are rules that limit the values that can be entered into a column or set of columns in a table. These rules are enforced by the database system, preventing the insertion or modification of rows that violate the specified conditions. Check constraints are defined using the CHECK keyword followed by an expression that evaluates to a Boolean value.

Validating Rental Durations

Let's consider a scenario using a modified version of the "rental" table in the "dvdrental" database that contains a "rental_duration" column. The table definition might appear as follows in the Navicat Table Designer:

Now, suppose we want to ensure that the duration of a rental is always greater than zero days. We can achieve this by adding a check constraint to the "rentals_with_rental_period" table as follows:

ALTER TABLE rentals_with_rental_periodADD CONSTRAINT rental_duration_checkCHECK (rental_duration > 0);    

In Navicat we can add a check constraint in the "Checks" tab of the Table Designer. We just need to supply an expression and optional name. Navicat will create a unique name for us if we don't supply one!

Upon hitting the Save button, Navicat will either create the check constraint or show an error message if any rows violate the constraint.

With this constraint in place, any attempt to insert or update a row in the "rentals_with_rental_period" table where the rental duration is less than or equal to zero will result in an error, ensuring that only valid rental durations are allowed.

Enforcing Valid Ratings

Another example from the "film" table in the "dvdrental" database involves validating film ratings. Suppose we want to restrict the ratings to only certain values, such as 'G', 'PG', 'PG-13', 'R', or 'NC-17'. We can achieve this with a check constraint:

ALTER TABLE filmADD CONSTRAINT film_rating_checkCHECK (rating IN ('G', 'PG', 'PG-13', 'R', 'NC-17'));    

Here is the same constraint in the Navicat Table Designer:

Now, any attempt to insert or update a row in the "film" table with a rating that is not one of the specified values will be rejected, ensuring that only valid ratings are allowed.

Handling NULL Values

It's important to note that check constraints are not applied to rows where one or more columns contain a NULL value unless the constraint specifically includes a condition to check for NULL. For example, to enforce that the "rental_rate" column in the "film" table is always greater than zero and not NULL, we would use the following constraint:

ALTER TABLE filmADD CONSTRAINT film_rental_rate_checkCHECK (rental_rate > 0 AND rental_rate IS NOT NULL);    

Here is the same constraint in the Navicat Table Designer:

Conclusion

Check constraints are a powerful tool for ensuring data integrity in PostgreSQL. By defining rules that data must adhere to, you can prevent the insertion or modification of invalid data, helping to maintain the accuracy and consistency of your database. By incorporating them into your database design, you can build robust and reliable data systems that meet the needs of your organization.

PostgreSQL, renowned for its robustness and extensibility, offers several helpful functions for both developers and database administrators alike. Among these functions, file_fdw_handler, file_fdw_validator, pg_stat_statements, pg_stat_statements_info, and pg_stat_statements_reset stand out as invaluable tools for enhancing database management and performance optimization. In today's blog we'll learn how to use all of these functions as well as how Navicat can help!

File Functions

PostgreSQL's Foreign Data Wrapper (FDW) functionality allows seamless integration of external data sources into the database. The file_fdw_handler and file_fdw_validator functions are specifically designed to handle foreign tables backed by files.

The file_fdw_handler function serves as an interface between PostgreSQL and the foreign data source, enabling the execution of SQL queries against files residing outside the database. Let's consider an example where we want to create a foreign table named external_data referencing a CSV file named data.csv:

        CREATE SERVER file_server FOREIGN DATA WRAPPER file_fdw;        CREATE FOREIGN TABLE external_data (            id INT,            name TEXT,            age INT        ) SERVER file_server OPTIONS (filename '/path/to/data.csv');    

Meanwhile, the file_fdw_validator function ensures the integrity of the options provided when creating a foreign table. It validates if the specified file exists and is accessible. For instance:

        SELECT file_fdw_validator('filename', '/path/to/data.csv');    

Statistical Functions

PostgreSQL's pg_stat_statements module provides a set of built-in functions for monitoring and analyzing query performance. Among these, pg_stat_statements, pg_stat_statements_info, and pg_stat_statements_reset are indispensable for identifying bottlenecks and optimizing database performance.

pg_stat_statements is a module that records statistics about SQL statements executed by a server. It tracks details such as execution counts, total runtime, and resource usage for each unique query. To enable pg_stat_statements, you need to add it to the shared_preload_libraries configuration parameter in postgresql.conf:

shared_preload_libraries = 'pg_stat_statements'

After restarting the PostgreSQL server, you can query the statistics using:

        SELECT * FROM pg_stat_statements;    

pg_stat_statements_info provides additional information about the pg_stat_statements module, such as the version number and the last reset time. It can be queried as follows:

        SELECT * FROM pg_stat_statements_info;    

Finally, pg_stat_statements_reset resets the statistics collected by pg_stat_statements, allowing you to start afresh with performance monitoring. Simply execute:

        SELECT pg_stat_statements_reset();    

Working with PostgreSQL's Built-in Functions in Navicat

We can access all of the above functions in Navicat for PostgreSQL or Navicat Premium 16 by expanding the "Functions" section in the Objects Pane:

To execute a function, simply select it from the Objects list and click the Execute Function button:

That will bring up a dialog where you can supply input parameter values:

Click the OK button to execute the function and view the results (or Cancel to abort):

PostgreSQL's built-in functions, including file_fdw_handler, file_fdw_validator, pg_stat_statements, pg_stat_statements_info, and pg_stat_statements_reset, play a pivotal role in enhancing database management and optimizing query performance. By leveraging these functions effectively, developers and administrators can streamline operations and ensure optimal utilization of PostgreSQL's capabilities.

Range Types

Range types offer a concise way to represent a range of values within a single database field. They find application in various domains, from temporal data to numeric intervals. In this blog article, we'll be delving into their usage (and benefits!) using both DML/SQL statements and Navicat for PostgreSQL 16.

Understanding Range Types

In PostgreSQL, range types allow for the representation of continuous ranges of values. These ranges can be of different data types such as numeric, date, or timestamp. For example, a range might represent a period of time, a set of temperatures, or a range of product prices.

Practical Example: Tracking Rental Durations

Let's consider a scenario where we want to track the duration of rentals in the free dvdrental sample database. We can utilize range types to store rental durations efficiently. Here are the statements to create and populate the new "rentals_with_rental_period" table:

CREATE TABLE rentals_with_rental_period (    rental_id SERIAL PRIMARY KEY,    customer_id INT,    rental_duration INT,    rental_period DATERANGE);INSERT INTO rentals_with_rental_period (customer_id, rental_duration, rental_period)VALUES(1, 7, '[2024-02-01, 2024-02-08]'),(2, 5, '[2024-01-15, 2024-01-20]');    

In Navicat, we can create our table using the Table Designer:

After creating the table, we can add data to it. Be sure to prefix the Range values with a square bracket "[" and end them with a parenthesis ")". That tells Navicat that the values belong to a range:

In this example, the "rental_period" column stores ranges representing the start and end dates of each rental. We can easily query rentals that include a specific date using the @> operator:

Expanding Applications: Numeric Intervals

Range types are not limited to temporal data. They can also be used to represent numeric intervals. For instance, imagine a scenario where a product's price can vary within a specific range based on quantity purchased. We can use range types to model this effectively:

CREATE TABLE product_price (    product_id SERIAL PRIMARY KEY,    price_range NUMRANGE);INSERT INTO product_price (price_range)VALUES('[10.00, 20.00)'),('[20.00, 30.00)'),('[30.00, )');    

In this example, the "price_range" column stores ranges representing the minimum and maximum prices for each product. We can query products within a specific price range using the @> operator:

SELECT * FROM product_priceWHERE price_range @> 25.00;    

Conclusion

Range types in PostgreSQL offer a powerful way to represent and query continuous ranges of values. Whether dealing with temporal data, numeric intervals, or other continuous values, range types provide a concise and efficient solution. By leveraging range types, developers can enhance the expressiveness and flexibility of their database schemas, paving the way for more sophisticated applications.

Looking for an easy-to-use graphical tool for PostgreSQL database development? Navicat 16 For PostgreSQL has got you covered. Click here to download the fully functioning application for a free 14 day trial!

Arrays and Enums

PostgreSQL, renowned for its extensibility and versatility, offers several data types beyond the conventional integer and string. Among these are the array and enum, which empower developers with advanced data modeling capabilities. In this blog article, we'll be delving into these sophisticated data types, demonstrating their usage and benefits within the context of the free dvdrental sample database.

The Array Type

Arrays in PostgreSQL enable the storage of multiple values within a single database field. This capability proves invaluable in scenarios where dealing with lists or sets of data is essential. Let's consider a practical example. Suppose we want to store films along with the actors who appeared each film. We can utilize the array data type to achieve this efficiently. First, here are the statements to create and populate the new "films_with_actors" table:

CREATE TABLE films_with_actors (    film_id SERIAL PRIMARY KEY,    title VARCHAR(255),    actors TEXT[]);INSERT INTO films_with_actors (title, actors) VALUES('Inception', ARRAY['Leonardo DiCaprio', 'Joseph Gordon-Levitt']),('The Shawshank Redemption', ARRAY['Tim Robbins', 'Morgan Freeman']);    

In Navicat, we can create our table using the Table Designer:

Note that if we append square brackets "[]" to the text type Navicat will recognize it an Array type and add a "1" to the Dimension field upon saving the table, indicating that it is a one-dimensional array.

After creating the table, we will be able to add data to it. Be sure to enclose the Array values in curly braces "{}" to tell Navicat which values to include within each array:

In queries, we can refer to a specific Array element by appending the desired index within square brackets. Hence, "actors[1]" would fetch the first Array value:

The Enum Type

Short for "Enumerated", the Enum type allows developers to define a fixed set of possible values for a column. This enhances data integrity and clarity within the database schema. Let's illustrate this by adding a "rating" column to the "films_with_actors" table. We can define an enumerated type for movie ratings using the following DDL statement:

CREATE TYPE rating AS ENUM ('G', 'PG', 'PG-13', 'R', 'NC-17');ALTER TABLE films_with_actors ADD COLUMN rating rating;    

In Navicat, we can append the new column in the Table Designer by clicking the "Add Field" button above the column list. After we've created the rating Enum using the CREATE TYPE statement above, we can choose it by selecting the "(Type)" item from the Type drop-down and then choosing the rating item from the Object Type list:

The table rating column will now include a drop-down with our defined Enum values:

Conclusion

PostgreSQL's array and enum data types provide developers with powerful tools to model complex data structures efficiently. By leveraging these advanced features, developers can enhance data integrity, streamline queries, and build more robust database schemas. In next week's blog, we'll conclude our exploration of PostgreSQL's advanced data types with a look at the Range type. Offering a concise way to represent a range of values within a single database field, the Range type is highly useful in various domains, from temporal data to numeric intervals.

Looking for an easy-to-use graphical tool for PostgreSQL database development? Navicat 16 For PostgreSQL has got you covered. Click here to download the fully functioning application for a free 14 day trial!

In the dynamic landscape of database management systems, selecting the right platform for your project is a crucial decision. With an array of options available, each catering to specific needs, making a choice can be a daunting task. This blog will outline a few reasons why PostgreSQL may just be the relational database solution you're looking for.

Open Source Advantage

At the core of PostgreSQL's appeal is its open-source nature. Open-source databases offer a cost-effective solution without compromising on features and performance. With PostgreSQL, you benefit from a vibrant community of developers constantly improving and refining the system. This collaborative effort ensures that the database stays up-to-date with the latest technological advancements and security measures.

Extensibility and Customization

PostgreSQL's extensibility sets it apart from many of its counterparts. It allows users to define their own data types, operators, and functions, giving developers a high degree of flexibility in tailoring the database to their specific project requirements. This extensibility is a boon for projects with unique data storage and processing needs.

Advanced Data Types and Features

PostgreSQL supports a wide range of advanced data types, including arrays, hstore (key-value pairs), and JSON. Its support for complex data structures makes it an ideal choice for projects that demand versatility and adaptability in handling diverse data formats. Additionally, features like full-text search, geospatial support, and advanced indexing mechanisms enhance its capability to manage complex datasets efficiently.

ACID Compliance

PostgreSQL adheres strictly to the ACID (Atomicity, Consistency, Isolation, Durability) principles, ensuring transactional integrity even in the most demanding environments. This level of reliability is crucial for applications where data consistency and accuracy are non-negotiable, such as financial systems or healthcare applications.

Performance Tuning and Optimization

PostgreSQL provides a wide assortment of performance tuning options, allowing developers to optimize the database for specific workloads. Its query optimizer is renowned for its efficiency, and administrators can fine-tune various parameters to achieve optimal performance tailored to their specific requirements. Whether handling large-scale data warehousing or real-time analytics, PostgreSQL can be fine-tuned to deliver exceptional speed and responsiveness.

Robust Community Support

The PostgreSQL community is one of the most active and supportive in the open-source database realm. With a vast pool of experienced developers, administrators, and contributors, users can easily find solutions to challenges, share best practices, and stay updated on the latest developments. The community-driven approach ensures a wealth of resources, including documentation, forums, and third-party tools, contributing to a smooth development and maintenance process.

Scalability

Scalability is a critical factor for any growing project. PostgreSQL excels in that department, supporting both vertical and horizontal scaling. Whether your project demands a single-node deployment or a distributed architecture, PostgreSQL can seamlessly adapt to varying workloads and data volumes, ensuring your database can evolve alongside your project's needs.

Conclusion

In today's blog, we explored a few reasons why PostgreSQL may be the right relational database solution for your next IT project. Its open-source nature, extensibility, advanced features, ACID compliance, performance tuning capabilities, robust community support, and scalability make it an ideal solution for projects ranging from small-scale applications to large enterprise systems.

Looking for an easy-to-use graphical tool for PostgreSQL database development? Navicat 16 For PostgreSQL has got you covered. Click here to download the fully functioning application for a free 14 day trial!

Last week's tutorial guided us through the creation of Materialized Views in PostgreSQL, using the DVD Rental Database as a practical example. As we learned there, PostgreSQL Materialized Views provide a powerful mechanism to enhance query performance by precomputing and storing the result set of a query as a physical table. Today's follow-up will cover other pertinent Materialized View operations such as refreshing a view, executing queries against it, as well as deleting a view should you no longer require it. As with the last blog article, we'll go over both DML statements as well as how to achieve the same result via the Navicat GUI.

Refreshing a Materialized View

The data in a Materialized View needs to be refreshed periodically to reflect any changes in the underlying tables. You can use the following command to refresh the Materialized View:

REFRESH MATERIALIZED VIEW mv_category_revenue;

In Navicat, we can refresh and completely replace the contents of a materialized view by right-clicking (or control-click on macOS) it in the Objects tab and selecting "Refresh Materialized View With" -> "Data" or "No Data" from the pop-up menu:

Querying a Materialized View

Now that we have our Materialized View, we can query it just like any other table:

SELECT * FROM mv_category_revenue;

This query will return the film categories along with their total revenue, providing a quick and efficient way to retrieve this information without repeatedly joining multiple tables.

In Navicat, you can write a query in the Query Editor or using the Query Builder tool. In the case of the Query Editor, the autocomplete feature will recognize the Materialized View with just a couple of key strokes!

Materialized Views are also included in the Object pane of the Query Builder. You can add a Materialized View to the query by dragging it from the Object pane to the Diagram pane or by double-clicking it on the Object pane:

Deleting a Materialized View

Should you no longer require a Materialized View, you can delete it using the DROP MATERIALIZED VIEW command. Here's the statement to drop the mv_category_revenue view:

DROP MATERIALIZED VIEW mv_category_revenue;

There are a couple of ways to delete a Materialized View in Navicat. The first is to:

• Select "Materialized View" in the Main Window's toolbar.
• Then select the Materialized View that you want to delete in the Objects list. That will enable several buttons on the Objects toolbar, including the Delete Materialized View button:

  

• Clicking the Delete Materialized View button will present a dialog prompt where you can confirm that you really do wish to delete the selected Materialized View.

The second way to delete a Materialized View in Navicat is to right-click it (or control-click on macOS) in either the Main Window's Navigation pane or Objects list and select "Delete Materialized View" from the context menu:

Conclusion

In this tutorial, we learned how to execute some pertinent Materialized View operations including refreshing a view, executing queries against it, as well as deleting a view. In each case, we covered both DML statements as well as how to achieve the same result via the Navicat GUI.

PostgreSQL Materialized Views provide a powerful mechanism to enhance query performance by precomputing and storing the result set of a query as a physical table. This tutorial will guide you through the creation of Materialized Views in PostgreSQL, using the DVD Rental Database as a practical example.

Understanding Materialized Views

A Materialized View is a snapshot of a query's result set that is stored as a physical table. Unlike regular views, which are virtual and execute the underlying query every time they are referenced, Materialized Views persist the data, allowing for faster query performance at the cost of periodic refreshes.

Materialized Views are particularly useful in scenarios where the underlying data changes infrequently compared to the frequency of query executions. This makes them ideal for scenarios such as reporting, data warehousing, and situations where real-time data is not a strict requirement.

Setting Up the DVD Rental Database

Before we dive into Materialized Views, let's set up the DVD Rental Database. It's PostgreSQL's version of the popular Sakila Sample Database for MySQL. You can download the DVD Rental Database from the official PostgreSQL tutorial page (PostgreSQL Sample Database).

The database file is in ZIP format (dvdrental.zip) so you need to extract it to dvdrental.tar before loading the sample database into the PostgreSQL database server. Once you have extracted the .tar file, create a new database called "dvdrental" and execute the pg_restore command to populate the dvdrental database from the contents of the .tar file:

        pg_restore -U postgres -d dvdrental D:\sampledb\postgres\dvdrental.tar    

Replace the path above with the one that points to the extracted dvdrental.tar on your system.

You can find the detailed installation instructions here.

Creating a Materialized View

Let's say we want to create a Materialized View that shows the total revenue generated by each film category. Here's a step-by-step guide:

• Connect to your PostgreSQL database
• Create the Materialized View using the following DML statement:

CREATE MATERIALIZED VIEW mv_category_revenue ASSELECT    c.name AS category,    SUM(p.amount) AS total_revenueFROM    category c    JOIN film_category fc ON c.category_id = fc.category_id    JOIN film f ON fc.film_id = f.film_id    JOIN inventory i ON f.film_id = i.film_id    JOIN rental r ON i.inventory_id = r.inventory_id    JOIN payment p ON r.rental_id = p.rental_idGROUP BY    c.name;

In this example, we join multiple tables from the DVD Rental Database to calculate the total revenue for each film category.

In Navicat For PostgreSQL (or Navicat Premium) 16:

• Click the "Materialized View" button to show the Materialized View Object List and then click on "+ New Materialized View" in the Objects toolbar to open the View Designer:

  

• Enter the SELECT portion of the above statement into the Definition editor:

  

• We can click the Preview button to verify that our statement works as expected:

  

• To create the new Materialized View, click the Save button. A dialog will appear prompting for the Materialized View Name. Let's call it "mv_category_revenue" just as we did in the above CREATE MATERIALIZED VIEW statement above:

  

• Upon clicking the dialog Save button, Navicat will change the new materialized view name from "untitled" to the one we provided. It will also add our new materialized view to the Materialized Views in the left-hand Navigation Pane:

  

Conclusion

PostgreSQL Materialized Views are a valuable tool for optimizing query performance in scenarios where real-time data is not critical. By pre-computing and storing the results of complex queries, Materialized Views can significantly improve response times for analytical and reporting tasks. In this tutorial, we learned how to create a Materialized View for the DVD Rental Database, showcasing their practical application in a real-world scenario.

SQLite is a lightweight, self-contained, and serverless relational database management system (RDBMS) that is widely used for embedded systems, mobile applications, and small to medium-sized websites. It is easy to set up, requires minimal configuration, and offers a powerful set of features for managing and manipulating data. In this guide, we will walk you through the process of getting started with SQLite, including installation and using the popular Chinook sample database for SQL examples.

Installing SQLite

Windows:

• Visit the SQLite download page at https://www.sqlite.org/download.html.
• Scroll down to the "Precompiled Binaries for Windows" section.
• Download the appropriate precompiled binary for your system architecture (32-bit or 64-bit).
• Extract the downloaded ZIP file to a location on your machine.
• Open the extracted folder and locate the sqlite3.dll executable.
• To make SQLite accessible from any command prompt window, add the folder containing sqlite3.dll to your system's PATH environment variable.

macOS:

• SQLite is pre-installed on macOS, so there's no need for a separate installation.
• Open the Terminal application.
• Type sqlite3 and press Enter to start the SQLite shell.

Linux:

• Most Linux distributions come with SQLite pre-installed. If not, you can install it using your package manager.
• For Debian/Ubuntu: sudo apt-get install sqlite3
• For Red Hat/Fedora: sudo dnf install sqlite
• For Arch Linux: sudo pacman -S sqlite
• Once installed, open the terminal and type sqlite3 to start the SQLite shell.

Using the Chinook Sample Database in Navicat

• Download the Chinook database ZIP file and extract its contents.

  You will find a file named chinook.db. Let's create a new database connection in Navicat.

• Select File -> New Connection -> SQLite... from the main menu to launch the New Connection dialog:

  

• In the dialog, enter "Chinook" for the Connection Name and then click on the Ellipsis button [...] to navigate to the Database File. Click the Test Connection to verify that we can connect to the database. (Note that the Chinook database does not require a user name or password):

  

• Click the OK button to close the dialog. You should see our new connection in the Connections pane:

  

Basic SQL Operations with Chinook

Connecting to the Chinook Database:

Now that we've created a new connection for the Chinook database, let's open the connection so that we can interact with the database. To do that:

• Locate the Chinook item in the Connections Pane and click on it in order to highlight it.
• Select File -> Open Connection from the main menu. That should show the main database.

Querying Data

To retrieve information from the Chinook database, you can use the SELECT statement. For example "SELECT * FROM artists;":

Filtering Data

Filtering allows you to narrow down your results. For instance, try "SELECT trackid, name, composer FROM tracks WHERE composer = 'Ludwig van Beethoven';":

Updating Records

To update existing data, we can use the UPDATE statement, or simply edit the data in place!

Inserting Records

To add a new record, no need to use the INSERT statement; in Navicat, we can simply click the Add Record button:

That will append a new empty row to the table, ready for data entry:

Deleting Records

Deleting a record in Navicat is equally straightforward; just highlight the row to remove and click the Delete key. A dialog will appear, asking for confirmation:

In today's blog, we learned how to get started with SQLite, including the installation process and how to perform basic SQL operations against the popular Chinook sample database. Whether you're a beginner or an experienced developer, SQLite's simplicity and versatility make it an excellent choice for various applications. Moreover, Navicat for SQLite (or Navicat Premium) 16 is the perfect tool to explore SQLite's more advanced features and capabilities and to efficiently manage your data.

Navicat Monitor 3 is a safe, simple and agentless remote server monitoring tool that includes many powerful features to make your monitoring effective as possible. You can access Navicat Monitor from anywhere via a web browser to access statistics on server load and performance regarding its availability, disk usage, network I/O, table locks and more.

Did you know that you can also collect custom performance metrics for specific instances using your own query, and receive alerts about your custom data when the metric value passes certain thresholds and durations? Custom metrics can even be displayed as charts to help better understand your data and quickly identify trends. In today's blog we'll create a custom metric that shows the average cost of movie rentals in the Sakila Sample Database.

Creating a Custom Metric

You'll find the Custom Metrics page in the ALERT & REPORT section on the Configurations tab:

• In the Custom Metrics page, click the "+ New Custom Metric" button to bring up the New Custom Metric dialog.
• The first step is to provide the METRIC DETAILS, which include the METRIC NAME and DESCRIPTION. We'll call our metric "Average Payment". (Note that the name cannot include the "<", ">", ":", """, "/", "|", "?", "*" characters or any OS reserved names.) For the DESCRIPTION, let's add "The average payment in the Sakila payment table.".
• For the DATABASE TYPE, we'll select "MySQL".
• We can collected metrics from all instances to one specific database. In our case, we'll select our MySQL instance.
• In the QUERY field, it's important to note that the query must return a single, numeric scalar or NULL value. Therefore, we should, apply an aggregate function to the column of interest, such as AVG, MIN, or MAX. In our case, we'll use the AVG function: "SELECT AVG(amount) FROM sakila.payment;". Be sure to prefix the table with the DB name and then run the query in the Navicat client to make sure that it returns a single value.
• Now let's click the Test Metric Collection button to verify that data can be successfully collected from selected instances within a reasonable duration. Here's what our query produced:

  

• For the DATA DISPLAY we can choose to use collected or calculated values. Collected Values are the actual values collected after running the query whereas calculated values use a calculated rate of change between collections, which measure the difference of the metric value divided by the number of seconds between each collection. The latter is useful in situations where new values are collected very frequently. We'll stick with the Collected Values for our metrics.

  Here's what we've got so far:
  

• On the Next screen, we can add an alert for our Custom Metric. We might want to do so if our metric was related to the server's health, but since ours is really only informational, we'll move the Enable Alert slider to the off position. That will grey out the rest of the screen, with the exception of the ALERT NAME field, which is required. We'll call ours "Average Payment Alert":

  

• The next and final screen shows a summary of our new Custom Metric. There, we can ENABLE (or disable) DATA COLLECTION a well as ENABLE (or disable) ALERTs:

  

• Clicking the Create Custom Metric button will close the dialog and show our new Custom Metric in the list:

  

Conclusion

Navicat Monitor 3's Custom Metrics are the perfect tool for tracking data that is meaningful to you and your organization. Moreover, by viewing changes over time in an area or line chart allow you to better spot helpful patterns. Finally, alerts can inform you of potential opportunities or dangers as soon as possible so that you can respond in a timely fashion.

In the digital age, databases play a vital role in managing and organizing information for countless applications and systems. As stewards of valuable data, it's essential for businesses and developers to be mindful of the types of information stored in their databases. While databases are designed to efficiently handle data, there are certain types of information that should almost never be stored in a database. In this article, we'll explore three things you should avoid storing in your database to maintain data integrity, security, and compliance.

1. Duplicate and Redundant Data

Storing duplicate or redundant data in your database might seem harmless at first, but it can lead to a host of issues down the line. Duplicate data not only wastes storage space but also introduces the risk of inconsistencies and errors. When information is duplicated across multiple records, updating one instance may be overlooked, resulting in discrepancies that can compromise the accuracy of your data.

To address this, databases should be designed with normalization principles in mind. Normalization involves organizing data to minimize redundancy and dependency, ensuring that each piece of information is stored in one place. By doing so, you not only optimize storage but also enhance data consistency and integrity.

2. Credit Card Information

In the realm of online transactions and e-commerce, the protection of financial information is paramount. Storing credit card information in a database poses significant risks and raises serious concerns about compliance with industry standards such as the Payment Card Industry Data Security Standard (PCI DSS). The PCI DSS outlines strict guidelines for handling and securing credit card data to prevent fraud and protect consumers.

Rather than storing credit card information directly, businesses should leverage secure payment gateways. Payment gateways facilitate the secure transmission of credit card information between the customer, the merchant, and the financial institution. This not only reduces the risk of data breaches but also ensures compliance with industry regulations.

3. Sensitive Personal Identifiable Information (PII)

Storing sensitive personal information, such as social security numbers, passport details, or driver's license numbers, in a database without proper safeguards is a recipe for disaster. PII is a prime target for identity theft and can be misused for fraudulent activities if it falls into the wrong hands. Even if encryption is applied to the database, the risk remains high, as decryption keys could potentially be compromised.

To mitigate this risk, it's advisable to implement tokenization or pseudonymization techniques for handling PII. Tokenization involves replacing sensitive data with unique tokens, rendering the original information unreadable. Pseudonymization involves replacing or encrypting sensitive identifiers with reversible algorithms, ensuring data protection while maintaining usability for authorized users.

Exceptions and Considerations

While some values may be derived from other fields, exceptions may be made for performance reasons. In cases where you have millions of records, it's sometimes preferable to fetch the actual value from the database as opposed to cycling through records and dynamically calculating the answer every time. With that in mind, here are a couple of fields that you may want to store in your database:

Retail Price

The retail price of an item is often calculated as the cost plus the tax. However, this seemingly simple concept introduces complexities when underlying prices change or new sales taxes go into effect. Storing the calculated price in the database requires an 'as of date' along with it for context. This allows for a historical view of prices, ensuring accurate records even when factors affecting pricing change over time.

Age

Storing age information may seem unnecessary when you have someone's birthday and today's date. However, considering that age changes over time, storing the 'as of date' of the record and the 'as of age' at the time of storage eliminates the need for sometimes tricky calculations. This approach ensures that age-related information remains accurate, providing a snapshot of the individual's age at a specific point in time.

If you ever need to store a calculated field, you can quickly create an insert trigger in Navicat 16.

Conclusion

Knowing which pieces of data to include in your database is just as important as understanding what to exclude. By avoiding the storage of duplicate and redundant data, sensitive personal information, and certain types of information better suited for dynamic calculations, you not only optimize storage but also enhance data consistency, integrity, and security.

In today's digital age, data is the lifeblood of organizations. As such, securing that data has never been more crucial. To safeguard sensitive data from unauthorized access, breaches, and other security threats, it's essential to implement robust security measures. This article will offer a few such measures for securing your relational databases.

Access Control

One of the fundamental principles of database security is controlling access to stored data. Implementing strong access controls ensures that only authorized users can interact with the database. This involves assigning unique user accounts with specific permissions based on their roles and responsibilities.

• User Authentication: Enforce strong password policies and use multi-factor authentication to add an extra layer of security. This helps prevent unauthorized access, even if login credentials are compromised.
• Role-Based Access Control (RBAC): Implement RBAC to assign permissions based on users' roles within the organization. This minimizes the risk of users having unnecessary access to sensitive data.

Encryption

Encrypting data at rest and in transit is crucial to protect it from unauthorized access and interception.

• Data at Rest Encryption: Utilize encryption algorithms to secure data stored on disk or in backups. This prevents unauthorized access to the physical storage media, adding an additional layer of protection.
• Secure Sockets Layer (SSL) or Transport Layer Security (TLS): Encrypt data in transit by using SSL or TLS protocols. This ensures that communication between the database server and client applications is secure and cannot be easily intercepted.

Backup and Recovery

A solid backup and recovery strategy is essential to mitigate the impact of data loss or corruption due to security incidents.

• Regular Backups: Schedule regular backups of the database and verify their integrity. Store backups in a secure location separate from the production environment.
• Point-in-Time Recovery: Implement point-in-time recovery mechanisms to restore the database to a specific state before a security incident occurred.

Employee Training and Awareness

Human error is a common cause of security breaches. Educate employees about security best practices and the importance of safeguarding sensitive information. Conduct regular security awareness training to keep employees informed about the latest security threats and best practices. This empowers them to recognize and report potential security incidents.

Real-time Monitoring

Utilize real-time monitoring tools to track performance metrics and identify anomalies or potential security threats. That's where a tool such as Navicat Monitor 3 can help. It's a comprehensive database monitoring and performance optimization tool designed for MySQL, MariaDB, PostgreSQL and SQL Server, as well as cloud databases like Amazon RDS, Amazon Aurora, Oracle Cloud, Google Cloud and Microsoft Azure. Navicat Monitor 3 offers several features that contribute to the overall security of relational databases, including:

• Real-time Monitoring: Navicat Monitor 3 provides real-time monitoring of various performance metrics, including CPU usage, memory usage, and disk I/O. By staying informed about the database's health, administrators can quickly identify abnormal patterns that may indicate a security incident.
• Alerting and Notification: The tool offers customizable alerts and notifications for performance-related issues. By configuring alerts for specific thresholds, administrators can receive immediate notifications of potential security threats or unusual database activities.
• Historical Data Analysis: The ability to analyze historical performance data helps administrators identify patterns or trends that may indicate security issues. This feature enhances the proactive identification of potential threats before they escalate.

Conclusion:

Securing a relational database is a multifaceted task that requires a combination of access controls, encryption, monitoring, and proactive management. By implementing the security measures outlined in this article and leveraging tools like Navicat Monitor 3, you can fortify your organization's defenses against potential security threats and ensure the integrity and confidentiality of their valuable data.

MySQL, one of the most popular relational database management systems, offers a variety of storage engines, each designed to cater to specific needs and use cases. When it comes to optimizing your database performance and ensuring data integrity, selecting the right storage engine is crucial. In today's blog, we'll explore the key factors to consider when choosing a storage engine for your MySQL tables.

Understanding Storage Engines

MySQL supports multiple storage engines, each with its own set of features, strengths, and weaknesses. The storage engine is responsible for handling the storage, retrieval, and management of data in the database tables. While InnoDB and MyISAM are by far the most commonly used engines, there are several others to consider.

Consider Your Usage Patterns

The first step in choosing a storage engine is understanding your specific usage patterns. Different storage engines are optimized for various scenarios. Choices include:

• InnoDB: This is the default storage engine for MySQL and is well-suited for applications with high write-intensive workloads and transactions. InnoDB provides ACID compliance, ensuring data consistency and reliability.
• MyISAM: If your application has more read-intensive operations and doesn't require transactions, MyISAM might be a good choice. It performs well for scenarios like data warehousing and read-heavy reporting.
• MRG_MyISAM: A merge storage engine that allows you to create tables that are a collection of other MyISAM tables. Useful for managing large datasets spread across multiple tables.
• MEMORY: This storage engine stores all data in RAM, making it ideal for scenarios where fast access to data is critical. However, it's important to note that data stored in the MEMORY engine is volatile and doesn't persist across server restarts.
• Blackhole: Acts as a "black hole" where data is accepted but not stored. Useful for scenarios where you want to replicate data to other servers without actually storing it locally.
• CSV: Stores data in text files using the CSV format. Suitable for data exchange between databases and applications that use CSV files.
• Performance_Schema: A storage engine that provides performance-related information about server execution at runtime. Helpful for monitoring and optimizing performance.
• ARCHIVE: This engine is optimized for storing large amounts of data with minimal space requirements. It's suitable for archiving purposes where fast data retrieval is not a primary concern.

Comparing InnoDB to MyISAM

Since InnoDB and MyISAM are the most popular storage engines, let's take a moment to consider each engine's strengths and weaknesses in terms of transactional capabilities, data integrity, reliability, and performance.

If your application involves complex transactions and requires features like rollbacks and savepoints, InnoDB is a strong candidate. It provides full ACID compliance, ensuring that transactions are handled reliably. On the other hand, if your application doesn't rely heavily on transactions and can tolerate occasional data inconsistencies, a storage engine like MyISAM may be more suitable. MyISAM doesn't support transactions to the same extent as InnoDB, but it can perform well for read-heavy workloads.

For applications where data integrity is paramount, InnoDB is often the preferred choice. InnoDB uses a clustered index and supports foreign key constraints, ensuring referential integrity between tables. This is crucial for applications where maintaining data consistency is a top priority. If your application can tolerate a lower level of data integrity, MyISAM might be considered. MyISAM doesn't support foreign key constraints and is more prone to table-level corruption in the event of a crash. Therefore, it's essential to weigh the trade-offs between performance and data reliability.

Performance is a critical factor in choosing a storage engine. InnoDB is known for its excellent performance in write-intensive scenarios due to its support for multi-version concurrency control (MVCC). It uses row-level locking, reducing contention and allowing for better concurrency. MyISAM, on the other hand, excels in read-intensive workloads. It uses table-level locking, which can impact concurrency in write-heavy scenarios but allows for faster read operations.

Selecting a Storage Engine in Navicat

Since each table can have its own storage engine in MySQL, Navicat displays it in the Table Objects Explorer, along with other pertinent information, such as the latest Auto Increment Value, last Modified Date, Data Length, and number of Rows:

To set or change a table's storage engine, open the Table Designer and click on the Options tab. There you'll find a drop-down of supported Engines as well as a number of other relevant fields:

Different storage engines come with their own attributes, so the other configurable options will depend on the Engine you choose. For example, here are the fields for the InnoDB Engine:

Meanwhile, the MEMORY Engine offers fewer configuration options:

Conclusion

Selecting the appropriate storage engine for your MySQL tables is a critical decision that directly impacts your application's performance, reliability, and scalability. By carefully considering your usage patterns, transactional requirements, data integrity needs, performance considerations, and exploring specialized storage engines, you can make an informed decision that aligns with your organization's goals.

Navicat, a powerful database management tool, offers a robust environment for developers and database administrators alike. One of its key features is the SQL Editor, where users can write and execute SQL queries. The Editor settings in Navicat allow users to tailor their working environment to meet specific personal and organizational preferences. In this blog article, we'll explore the various configuration options available in Navicat's SQL Editor.

Accessing the Editor Settings

All of the configuration options for Navicat SQL Editors are conveniently located in one place. You'll find them on the Editor screen of the Options dialog. To bring up the Editor settings, click Tools -> Options... on the Main Menu to display the Options dialog and then select "Editor" in the left-hand pane:

Code Formatting

One of the first things you might want to customize in Navicat's SQL Editor is code formatting. Properly formatted code enhances readability and makes it easier to collaborate with team members. Navicat allows you to define indentation, font type and many other options.

To format the contents of the SQL Editor at any time, click the "Beautify SQL" button located on the Editor Toolbar.

Here's a before and after example:

Choosing a Color Scheme

Creating an appealing and clear color scheme can significantly improve your coding experience. Navicat allows you customize the color of common text, keywords, strings, numbers, comments, and even the background, allowing you to define a color scheme that aligns with your preferences.

To set a color, click on the square sample block to the right of the target property to open the system color dialog. There, you can choose a predefined color, saved custom color, or create a brand new custom color!

Here's a query using Courier font at 12pt with custom colors:

Highlighting Matching Brackets

Ensuring that matching brackets are easily identifiable is crucial for avoiding syntax errors. Navicat provides an option to highlight matching brackets, making it easier to navigate and troubleshoot your code.

Placing the cursor at the opening brace highlights both the opening and closing braces:

Disabling Syntax Highlighting

There are times that you may want to switch off syntax highlighting. This can be accomplished by deselecting the "Use syntax highlighting" checkbox. Syntax highlighting can also be turned off for large files by specifying a maximum MB value in the "Disable if file size is larger than (MB)" textbox.

Miscellaneous Options

Here are a few other settings that you're sure to find useful:

• Show line number: Display line numbers at the left side of the editor for easily reference.
• Use code folding: Code folding allows codes to collapse as a block and only the first line displayed in the editor.
• Tab Width: Enter the number of characters that a tab occupies, e.g. 5.

Final Thoughts on Configuring Editor Settings in Navicat

Customizing syntax highlighting in Navicat's SQL Editor is more than just about visual preferences; it's a practical way to enhance your coding efficiency and reduce the likelihood of errors. By carefully choosing colors, customizing styles, and enabling features like bracket highlighting, you can create a coding environment that aligns perfectly with your workflow. Take the time to explore these options and discover how they can elevate your SQL coding experience in Navicat!

While basic SQL queries can retrieve, insert, update, and delete records, aggregate queries take database interactions to a new level by providing the sums, averages, or min/max value from a large result set. In this article, we'll explore the fundamentals of aggregate SQL queries, examining how they can be employed to analyze and summarize data effectively.

Understanding Aggregate Functions

Aggregate functions in SQL operate on sets of rows and return a single value as output. These functions are invaluable when it comes to performing calculations on data within a database. Some of the commonly used aggregate functions include:

1. COUNT()

The COUNT() function tallies the number of rows that meet a specified condition. It can be used to count all rows or those satisfying certain criteria.

        SELECT COUNT(*) AS total_records FROM employees;    

2. SUM()

The SUM() function calculates the total sum of a numeric column.

        SELECT SUM(salary) AS total_salary FROM employees;    

3. AVG()

The AVG() function determines the average value of a numeric column.

        SELECT AVG(age) AS average_age FROM students;    

4. MAX() and MIN()

MAX() and MIN() functions identify the maximum and minimum values in a column, respectively.

        SELECT MAX(price) AS max_price, MIN(price) AS min_price FROM products;    

Grouping Data with GROUP BY

One of the powerful aspects of aggregate queries in SQL is the ability to group data based on certain criteria using the GROUP BY clause. This facilitates the analysis of subsets of data, allowing for more granular insights.

Grouping with COUNT()

        SELECT department, COUNT(*) AS employee_countFROM employeesGROUP BY department;    

Grouping with AVG()

        SELECT department, AVG(salary) AS average_salaryFROM employeesGROUP BY department;    

Filtering Groups with HAVING

The HAVING clause is used in conjunction with GROUP BY to filter the results of aggregate queries based on conditions applied to grouped data.

        SELECT department, AVG(salary) AS average_salaryFROM employeesGROUP BY departmentHAVING AVG(salary) > 50000;    

Combining Aggregate Functions

SQL allows for the combination of multiple aggregate functions in a single query, offering comprehensive insights into the data.

        SELECT department, COUNT(*) AS employee_count, AVG(salary) AS average_salaryFROM employeesGROUP BY department;    

Using Aggregate Functions in Navicat

If you're ever unsure of a function's exact name or input parameters, you can start typing it in the SQL Editor and Navicat will present a list of matching options that you can select from to autocomplete a term. Aggregate functions are identified by the Greek Sigma symbol (Σ):

Conclusion

Aggregate SQL queries are indispensable tools for data analysis and reporting in relational databases. Whether you're summarizing information, calculating averages, or grouping data based on certain criteria, understanding how to leverage aggregate functions and clauses like GROUP BY and HAVING is essential for proficient database querying.

In the realm of database optimization, understanding and monitoring query execution time is crucial. Whether you're a database administrator, developer, or involved in performance tuning, knowing how to measure the time a query takes to execute can provide valuable insights into the efficiency of your database operations. In this article, we'll explore various techniques for measuring query execution time in popular relational databases such as MySQL, PostgreSQL, and Microsoft SQL Server.

MySQL

Using SQL Profiler:

        SET profiling = 1;-- Your SQL Query Goes HereSHOW PROFILES;            

This sequence of commands enables profiling, executes your query, and then shows the profiling results. Look for the "Duration" column to find the execution time in seconds.

To calculate the total duration, you can use the following SQL query:

        SELECT SUM(Duration) AS TotalDurationFROM information_schema.profilingWHERE Query_ID > 1;            

PostgreSQL

Enabling Timing:

PostgreSQL has a built-in feature to measure query execution time. You can enable timing by executing the following command:

        \timing-- Your SQL Query Goes Here            

This will display the time taken to execute your query in milliseconds.

Using pg_stat_statements:

PostgreSQL comes with an extension called pg_stat_statements, which provides a detailed view of executed SQL statements. To use it, ensure the extension is enabled in your PostgreSQL configuration and execute the query:

        SELECT total_time, calls, queryFROM pg_stat_statementsWHERE query = 'Your SQL Query Goes Here';            

This will give you information about the total time spent executing the specified query.

Microsoft SQL Server

Using SET STATISTICS TIME:

SQL Server allows you to enable time statistics for a session using the SET STATISTICS TIME ON command. After executing your query, you'll receive a message in the "Messages" tab showing the total time:

        SET STATISTICS TIME ON-- Your SQL Query Goes HereSET STATISTICS TIME OFF            

Querying sys.dm_exec_query_stats:

For a more programmatic approach, you can query the sys.dm_exec_query_stats dynamic management view to get information about query execution times:

        SELECT total_elapsed_time, execution_count, textFROM sys.dm_exec_query_statsCROSS APPLY sys.dm_exec_sql_text(sql_handle)WHERE text LIKE 'Your SQL Query Goes Here%';            

This query retrieves information about the total elapsed time and the number of times the query has been executed.

Oracle Database

Using SQL*Plus AUTOTRACE:

Oracle Database provides the SQL*Plus AUTOTRACE feature, which can be used to display execution plans and statistics for SQL statements. To enable it, use the following commands:

        SET AUTOTRACE ON-- Your SQL Query Goes HereSET AUTOTRACE OFF            

The output will include information about the elapsed time, CPU time, and other statistics.

Querying V$SQL:

For more detailed information, you can query the V$SQL dynamic performance view:

        SELECT elapsed_time, executions, sql_textFROM V$SQLWHERE sql_text LIKE 'Your SQL Query Goes Here%';            

This query retrieves information about the elapsed time and the number of executions for the specified query.

Viewing Execution Time in Navicat

If you only need to view the total execution time of a query, you can find it at the bottom of the main Navicat application window, along with other pertinent query details:

Conclusion

Understanding and optimizing query execution time is fundamental to maintaining a high-performing database. By leveraging the tools and techniques discussed in this article, you can gain valuable insights into your database's performance and take proactive steps to enhance efficiency. Whether you're working with MySQL, PostgreSQL, Microsoft SQL Server, or Oracle Database, measuring and analyzing query execution time is a worthwhile endeavor for any database professional.

When it comes to selecting the right database for your application, the decision often boils down to the specific requirements of your project. Redis, a high-performance in-memory data store, and traditional relational databases such as MySQL, each offer their own strengths and weaknesses. In this guide, we will explore various factors to consider when deciding between Redis and a traditional relational database. For the sake of simplicity, we'll use MySQL as our traditional relational database. Should you decide to go that route, you may want to look at other relational database products such as SQL Server and Oracle.

Data Model and Structure

One of the primary differences between Redis and MySQL lies in their data models. Redis is a key-value store, where data is stored as pairs of keys and values. This simplicity makes it efficient for certain use cases like caching, session storage, and real-time analytics. On the other hand, as a relational database, MySQL allows you to define structured tables with relationships between them.

Hash Data in Redis

A MySQL Table

Consider your data structure and whether a key-value model or a relational model better suits your application's needs.

Performance

Redis is renowned for its exceptional performance, especially for read-heavy workloads and scenarios requiring low-latency responses. Being an in-memory database, Redis stores all data in RAM, providing fast access times. On the other hand, MySQL, while still performing well, might encounter bottlenecks as the dataset grows, especially in scenarios with complex queries and frequent write operations.

Example: Redis Read Operation

// Retrieving data from RedisredisClient.get("user:123", (err, result) => {    const userData = JSON.parse(result);    console.log(userData);});

Example: MySQL Read Operation

-- Retrieving data from the users table in MySQLSELECT * FROM users WHERE id = 123;

Consider the nature of your application's workload and whether the emphasis is on read or write operations.

Persistence

One key consideration is data persistence. Redis, being an in-memory store, may not be the best choice for scenarios where durability and persistence are critical. While Redis does offer persistence options, such as snapshots and append-only files, MySQL inherently provides more robust durability features.

Example: Redis Snapshot Persistence

// Configuring Redis to take snapshots every 5 minutesconfig set save "300 1";

Ensure your choice aligns with your application's requirements for data persistence.

Scalability

Scalability is another crucial factor. Redis excels in horizontal scalability, making it suitable for distributed setups and scenarios where you need to scale out across multiple nodes. MySQL, while also scalable, might require more effort and careful planning, especially in large-scale distributed environments.

Example: Redis Horizontal Scaling

// Creating a Redis cluster with three nodesredis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002

Example: MySQL Sharding

-- Sharding the users table across multiple databases-- (Assuming a sharding key 'user_id')CREATE TABLE users_shard_1 SELECT * FROM users WHERE user_id % 3 = 1;CREATE TABLE users_shard_2 SELECT * FROM users WHERE user_id % 3 = 2;CREATE TABLE users_shard_3 SELECT * FROM users WHERE user_id % 3 = 0;

Consider the scalability requirements of your application and whether your chosen database can scale accordingly.

Use Case Considerations

Understanding the specific use cases for Redis and MySQL is crucial for making an informed decision. With this in mind, here are the top three use cases of each database:

• Redis Use Cases:
  • Caching: Redis excels in caching due to its fast read access.
  • Real-time Analytics: Its in-memory nature is beneficial for quick data analysis.
  • Session Storage: Ideal for storing and managing session data.
• MySQL Use Cases:
  • Transactional Data: MySQL is well-suited for applications requiring ACID compliance.
  • Complex Queries: If your application involves complex queries and reporting, MySQL might be a better fit.
  • Data Integrity: For scenarios where relational data integrity is a priority.

Consider the specific requirements of your project and how well each database aligns with those needs.

Working with Redis

One reservation you may have about going with Redis is that its syntax is so dissimilar to that of traditional databases. However, that need not be an issue. Navicat for Redis, a powerful GUI tool designed to enhance the management and interaction with Redis databases, provides an intuitive interface for performing various tasks such as browsing, querying, and modifying data.

Conclusion

Choosing between Redis and MySQL involves careful consideration of factors such as data model, performance, persistence, scalability, and use case requirements. Assessing these aspects in the context of your application's specific needs will guide you toward the most suitable database for your project.

One of the most common questions Navicat Support receives from users is how to format dates and times in both Grid and form View. It's actually quite simple! In today's blog, we'll go over the steps to change date and time formats globally in Navicat Premium.

Where Navicat Defines Display Formats

You'll find display formats for dates and times on the Records screen of the Options dialog. It is accessible via the Tools -> Options... command from the main menu:

We can see the Date, Time, and DateTime formats in the Display Format section of the Records screen (highlighted with a red border):

Setting the Format

Let's go ahead and update the DateTime format using the Sakila Sample Database as an example. Many of its tables contain a DateTime field called last_update that is used for auditing purposes. We can see it in this screen capture of the actor table (again highlighted with a red border):

By default, Navicat displays dates and times in whatever format they are defined in the database. In the case of MySQL, it displays DateTime values in 'YYYY-MM-DD hh:mm:ss' format, for example '2019-10-12 14:35:18' (Notice the use of the 24 hour clock).

Standard SQL and ODBC Date and Time Literals

If you're unsure of the meaning of the letters in the 'YYYY-MM-DD hh:mm:ss' string, those are part of the Standard SQL and ODBC Date and Time Literals. These are standardized ways of representing date and time values in SQL queries. They provide a consistent and platform-independent method for specifying date and time values in SQL statements. Here's a list of each letter pattern and their meaning. You'll want to get acquainted with them because Navicat also uses them to set date and time formats:

• c - Display the date using the format given by the system global variable, followedby the time using the format given by the system global variable. The time is notdisplayed if the date-time value indicates midnight precisely.
• d - Display the day as a number without a leading zero (1-31).
• dd - Display the day as a number with a leading zero (01-31).
• ddd - Display the day as an abbreviation (Sun-Sat).
• dddd - Display the day as a full name (Sunday-Saturday).
• ddddd - Display the date using the format given by the system global variable.
• dddddd - Display the date using the format given by the system global variable.
• m - Display the month as a number without a leading zero (1-12). If the m specifierimmediately follows an h or hh specifier, the minute rather than the month isdisplayed.
• mm - Display the month as a number with a leading zero (01-12). If the mm specifierimmediately follows an h or hh specifier, the minute rather than the month isdisplayed.
• mmm - Display the month as an abbreviation (Jan-Dec) using the strings given by thesystem global variable.
• mmmm - Display the month as a full name (January-December) using the strings givenby the system global variable.
• yy - Display the year as a two-digit number (00-99).
• yyyy - Display the year as a four-digit number (0000-9999).
• h - Display the hour without a leading zero (0-23).
• hh - Display the hour with a leading zero (00-23).
• n - Display the minute without a leading zero (0-59).
• nn - Display the minute with a leading zero (00-59).
• s - Display the second without a leading zero (0-59).
• ss - Display the second with a leading zero (00-59).
• t - Display the time using the format given by the system global variable.
• tt - Display the time using the format given by the system global variable.
• am/pm - Use the 12-hour clock for the preceding h or hh specifier, and display 'am' forany hour before noon, and 'pm' for any hour after noon. The am/pm specifiercan use lower, upper, or mixed case, and the result is displayed accordingly.
• a/p - Use the 12-hour clock for the preceding h or hh specifier, and display 'a' for anyhour before noon, and 'p' for any hour after noon. The a/p specifier can uselower, upper, or mixed case, and the result is displayed accordingly.
• ampm - Use the 12-hour clock for the preceding h or hh specifier, and display thecontents of the system global variable for any hour before noon, and thecontents of the system global variable for any hour after noon.
• / - Date separator. In some locales, other characters may be used to represent thedate separator.
• : - Time separator. In some locales, other characters may be used to represent thetime separator.
• 'xx'/"xx" - Characters enclosed in single or double quotes are displayed as-is, with noformatting changes.

Now, let's change the global Navicat DateTime format to utilize the numeric day without a leading zero, the three letter month abbreviation, and the 12-hour clock with the AM/PM indicator.

Using the above instructions as our guide, that would give us a format string of "mmm d, yyyy hh:mm:ss AM/PM". We can see the results in real-time in the Output field as we type:

After closing the Options dialog via the OK button, all DateTime fields should now be using our custom DateTime format. Here is the last_update column of the actor table mentioned previously:

Remember that the new format will apply globally across all databases. To confirm this, let's take a look at the orders table from the classicmodels database. It contains three DateTime columns, but only sets the date portion. These columns also display their values according to our new format:

Conclusion

In this blog, we learned how to easily change date and time formats globally in the Options dialog. While we used Navicat Premium here today, note that other Navicat products, such as Navicat for MySQL or Navicat for SQL Server, would work in exactly the same way.

Navicat 16 came loaded with numerous improvements and features to address both database developers' and administrators' needs. With over 100 enhancements and a brand new interface, there are more ways than ever to build, manage, and maintain your databases. One of the many improvements aimed at maximizing productivity is the ability to configure multiple connection profiles. This feature is ideal for out-of-office users who may need to switch between settings based on their location or for streamlining database access. Today's blog will outline the process of creating a new connection profile and how to switch between profiles.

Connection Profiles at a Glance

Before demonstrating how to create a connection profile, let's briefly go over what they are. Simply put, Navicat connection profiles serve as predefined configurations that store all the necessary information needed to connect to a specific database. This includes details like the host address, port number, username, password, and the specific database you want to access. By creating and saving these profiles, you can quickly establish connections without manually entering these details every time.

Creating a New Connection Profile

When we create a connection to a database instance, it becomes the Main (active) Profile for the Open Connection command. However, if we open the New Connection or Edit Connection dialog we can see a button to toggle the Connection Profile pane in the lower left corner of the dialog:

There, we will see the Main Profile listed.

We can create a connection profile by clicking the New Connection Profile link at the bottom of the Connection Profile pane:

That will copy over the Main Profile details to the new profile and display a text field where we can assign it a name. We'll call it "Most Used Databases":

On the Databases tab, we'll deselect all but a few core databases:

Upon clicking the OK button, we are prompted by a dialog asking if we want to switch the Active Profile to the new one:

Let's go ahead and click "Switch". That will close the dialog and display our new Connection Profile beside the MySQL connection in the main Connections pane:

Now the Open Connection command will open our new Connection profile. We can easily confirm this because only the databases that we selected are shown in the Connections pane:

Switching Connection Profiles

Once a Connection Profile is open, we can easily switch to another via the Switch Connection Profile command. From there, we can select from any connection profile for that database instance or even create a new one!

A dialog prompt will remind us that the current connection will be closed before switching to the new one:

That will make the selected profile the Main one, so that Open Connection command will open it from now on.

Conclusion

Navicat 16's connection profiles are a powerful tool for simplifying database management tasks. By creating and organizing profiles, you can swiftly connect to databases without the need for manual entry of connection details. As you become more adept at using Navicat, you'll discover additional features and techniques that further enhance your database management experience.

Redis, the blazing-fast, in-memory data structure store, is revered for its prowess in handling key-value pairs. However, its utility extends far beyond basic key operations. In this article, we'll explore some of the most indispensable Redis commands (other than those which involve keys, since we've covered those proviously), unlocking the true potential of this powerful tool. We'll also see how to communicate directly with Redis from Navicat!

1. LPUSH and RPUSH

Redis's versatility shines through its ability to handle complex data structures. Two of the most powerful commands in this regard are LPUSH and RPUSH, which respectively add elements to the left and right ends of a list.

> LPUSH my_list "element1"(integer) 1> RPUSH my_list "element2"(integer) 2> LRANGE my_list 0 -11) "element1"2) "element2"

These commands are instrumental in scenarios where you need to manage ordered data sets.

2. LPOP and RPOP

To complement the list addition commands, Redis provides LPOP and RPOP, which respectively remove and return the first and last elements of a list.

> LPOP my_list"element1"> RPOP my_list"element2"

These commands are particularly useful when implementing queues or stacks.

3. SADD and SMEMBERS

Redis sets are collections of unique elements. SADD adds one or more members to a set, while SMEMBERS retrieves all the members of a set.

> SADD my_set "member1"(integer) 1> SADD my_set "member2"(integer) 1> SMEMBERS my_set1) "member1"2) "member2"

Sets are powerful for scenarios requiring membership testing or storing unique data.

4. ZADD and ZRANGE

Sorted sets in Redis provide an ordered collection of unique elements. ZADD adds elements with a specified score, while ZRANGE retrieves elements within a specified range.

> ZADD my_sorted_set 1 "element1"(integer) 1> ZADD my_sorted_set 2 "element2"(integer) 1> ZRANGE my_sorted_set 0 -1 WITHSCORES1) "element1"2) "1"3) "element2"4) "2"

Sorted sets are excellent for scenarios requiring ordered data retrieval.

5. HSET and HGET

Redis hashes are maps between string field names and string values. HSET sets the value of a field in a hash, while HGET retrieves the value of a field.

> HSET my_hash field1 "value1"(integer) 1> HSET my_hash field2 "value2"(integer) 1> HGET my_hash field1"value1"

Hashes are ideal for scenarios involving structured data.

6. PUBLISH and SUBSCRIBE

Redis excels not only in data storage but also in real-time messaging. The PUBLISH command allows a client to send a message to a channel, while the SUBSCRIBE command enables a client to listen to messages on a channel.

# Terminal 1> SUBSCRIBE my_channelReading messages... (press Ctrl-C to quit)1) "subscribe"2) "my_channel"3) (integer) 1# Terminal 2> PUBLISH my_channel "Hello, Redis!"(integer) 1

This feature is invaluable for building real-time applications and event-driven architectures.

7. SCAN

While not a command for direct data manipulation, the SCAN command is essential for iterating over keys in a Redis database without blocking the server. It provides a cursor-based approach to prevent overloading the system.

> SCAN 01) "0"2) 1) "my_list"   2) "my_set"   3) "my_sorted_set"   4) "my_hash"   5) "my_channel"

This command is crucial for operations involving large datasets.

Executing Commands in Navicat 16 for Redis

While you can accomplish practically everything you need to using Navicat intuitive GUI, you can issue commands directly to Redis via the Console window. It's accessible via the Tools->Console command on the main menu or the Console button on the main toolbar:

Here's some sample output produced by the SCAN command that we learned about above:

Final Thoughts on Redis Commands

Redis commands extend far beyond the key-value operations that we've explored in recent blog entries. By mastering these advanced commands for working with data structures, sets, sorted sets, hashes, and even real-time messaging, you can harness the full potential of Redis for a wide range of applications. Whether you're building a caching layer, implementing queues, or developing real-time applications, Navicat 16 for Redis provides a robust set of tools to meet your needs.

Redis, known for its blazing fast performance, is a versatile NoSQL database that excels in handling key-value pairs. While it's primarily designed for simple data structures, Redis also supports more complex data types like lists, sets, and even JSON documents. In this blog article, we'll delve into the world of JSON documents in Redis, exploring how to work with them both through the command-line interface (CLI) and with the help of Navicat 16 for Redis on macOS.

Understanding JSON in Redis

JSON (JavaScript Object Notation) is a widely used data interchange format that's both human-readable and machine-friendly. Redis introduced native support for JSON documents in version 6.0, allowing users to store, query, and manipulate JSON data efficiently.

JSON documents in Redis are stored as values associated with a specific key. This allows for easy retrieval and manipulation using Redis commands.

CLI: Interacting with JSON Documents

• Storing JSON Documents

  To store a JSON document in Redis, you can use the JSON.SET command:

  JSON.SET mykey . '{"name": "John Doe", "age": 30, "email": "john@example.com"}'

  In this example, we're storing a JSON object with a name, age, and email address under the key mykey.

• Retrieving JSON Documents

  Retrieving a JSON document is straightforward using the JSON.GET command:

  JSON.GET mykey

  This will return the JSON object associated with the key mykey.

• Updating JSON Documents

  Updating a JSON document can be done using the JSON.SET command again:

  JSON.SET mykey . '{"name": "John Doe", "age": 31, "email": "john@example.com"}'

• Querying JSON Documents

  Redis provides the JSON.GET command with a path argument for querying specific elements within a JSON document:

  JSON.GET mykey .name

  This will return the value of the name field.

• Deleting JSON Documents

  Removing a JSON document is as simple as deleting the key associated with it:

  DEL mykey

Using Navicat for Redis

While Redis CLI offers a command-line approach for working with JSON documents, using a graphical tool Navicat can significantly enhance the user experience, especially for those who prefer a more visual approach. Navicat for Redis (macOS) version 16.2.6 supports the JSON key type.

Navicat for Redis (macOS) version 16.2.6 Main Screen

• Connecting to Redis with Navicat

  • Launch Navicat and select Connection -> Redis... from the main Toolbar.
  • Fill in the connection details (Host, Port, Authentication if required).
  • Click "Save" to establish a connection.

• Navigating JSON Documents

  In Navicat, you can view and interact with Redis data in a structured manner. To work with JSON documents:

  • Locate the key containing the JSON document in the main "All Data" table.
  • Select the key and click the Editor button to view the key's value.

• Editing JSON Documents

  Navicat provides a user-friendly JSON editor. You can directly modify the JSON document and save the changes.

Final Thoughts on Working with JSON Documents in Redis

Redis' integration of JSON documents extends its capabilities beyond simple key-value pairs, opening up new possibilities for handling structured data. Whether you're a developer managing complex data structures or a data analyst querying JSON data, Redis provides a robust platform for your needs. Navicat 16 for Redis for macOS's intuitive interface will help you to navigate and manipulate JSON documents with unparalleled ease and efficiency. Its intuitive JSON editor makes Navicat an invaluable tool, particularly for those who prefer a more visual approach to database management.

Redis, short for Remote Dictionary Server, is a versatile and high-performance key-value store that has gained significant popularity in the world of databases. It is known for its exceptional speed and efficiency in handling simple data structures. In this article, we will explore what sets Redis apart from other databases and how Navicat for Redis complements it as a robust management tool.

Speed and Simplicity

Redis distinguishes itself with its remarkable speed, primarily owing to its in-memory nature. Unlike traditional databases that rely on disk storage, Redis stores data in RAM, enabling lightning-fast read and write operations. This makes Redis an ideal choice for applications that require quick data retrieval and low latency.

For example, consider a use case where a social media platform needs to retrieve user profile information. With Redis, this operation is executed almost instantaneously due to the in-memory storage, eliminating the delays associated with disk I/O operations.

Data Structures for Flexibility

One of Redis's strengths lies in its support for a wide range of data structures, each tailored for specific use cases:

• Strings: Basic key-value pairs that can store strings, integers, or floating-point numbers.
  Example:
  SET user:1 "John Doe"
• Lists: Collections of ordered elements allowing push and pop operations from both ends.
  Example:
  LPUSH mylist "item1"
• Sets: Unordered collections of unique elements, useful for tasks like counting unique items or creating tag systems.
  Example:
  SADD tags "Redis" "Database" "NoSQL"
• Hashes: Maps between string field and string values, perfect for representing objects.
  Example:
  HSET user:1 username "johndoe" email "john@example.com"

These data structures empower developers to select the most suitable structure for their specific use case, resulting in optimized performance.

Pub/Sub Messaging

Redis offers robust support for Publish/Subscribe messaging, enabling real-time communication between different parts of an application or even between different applications. This feature is invaluable in scenarios requiring instant updates or notifications.

For example, in a gaming application, Redis Pub/Sub can be employed to notify players about in-game events, such as a new message or a player joining a room.

PUBLISH game:updates "New message: Hello, world!"

Lua Scripting for Complex Operations

Redis provides the ability to execute Lua scripts, allowing developers to perform complex operations in a single command. This is particularly useful for tasks that involve multiple steps or conditional logic.

For example, suppose you need to atomically transfer funds from one account to another while ensuring consistency. This can be accomplished with Lua scripting.

local sender_balance = tonumber(redis.call('GET', KEYS[1]))local receiver_balance = tonumber(redis.call('GET', KEYS[2]))local amount = tonumber(ARGV[1])if sender_balance >= amount then    redis.call('DECRBY', KEYS[1], amount)    redis.call('INCRBY', KEYS[2], amount)    return "SUCCESS"else    return "INSUFFICIENT FUNDS"end

Navicat for Redis: A Comprehensive Management Tool

Navicat for Redis is a powerful GUI tool designed to enhance the management and interaction with Redis databases. It provides an intuitive interface for performing various tasks such as browsing, querying, and modifying data. Here are some key features that set Navicat for Redis apart:

• User-Friendly Interface: Navicat for Redis offers an intuitive and user-friendly interface, making it easy for both novice and experienced developers to navigate and interact with Redis databases.
• Visual Data Manipulation: With Navicat, users can easily view, edit, and manipulate data within Redis databases. This is particularly useful for tasks like updating values or adding new keys.
• Query Building: The tool allows users to construct and execute complex queries using a graphical interface. This can be a significant time-saver for developers who prefer a visual approach to querying.
• Data Import and Export: Navicat supports seamless data import and export operations, facilitating tasks such as migrating data between databases or creating backups.
• Task Automation: Navicat for Redis enables the scheduling and automation of routine tasks, helping to streamline database management processes.

Final Thoughts on What Sets Redis Apart from Other Databases

Redis stands out as a high-performance key-value store, thanks to its in-memory nature and versatile data structures. It excels in scenarios where speed and low latency are paramount. The addition of Navicat for Redis enhances the Redis experience by providing a user-friendly and efficient management tool. Navicat's features like visual data manipulation, query building, and task automation make it a valuable companion for developers working with Redis databases. Together, Redis and Navicat form a powerful combination for building robust and high-performing applications.

Navicat made headlines back in May of 2023 when the company introduced Navicat for Redis. Since then, the development team have added several notable enhancements, the most significant being support for the Redis JSON key type. Version 16.3 marks another milestone in the evolution of both Navicat Premium and Navicat for Redis, which adds support for Redis Cluster. Today's blog will provide a brief overview of Redis Cluster, how to connect to server instances in Navicat, as well as list a few other features that you'll find in Navicat Premium.

Redis Cluster 101

Redis Cluster is a distributed implementation of Redis, the popular in-memory data structure store. It brings high availability and scalability to Redis setups. Introduced in Redis 3.0, it has become a crucial tool for large-scale applications.

One of its key features is automatic data sharding. Redis Cluster partitions the dataset across nodes, allowing for horizontal scaling. Each node holds a specific range of hash slots. This enables the handling of larger datasets compared to a single Redis instance.

Moreover, Redis Cluster ensures high availability through a master-slave replication model. Data is replicated across nodes, providing resilience against node failures. In case of a failure, a failover mechanism promotes a replica to a master, ensuring uninterrupted access to data.

Redis Cluster prioritizes availability and partition tolerance, making it a robust choice for distributed systems. It provides a balance between scalability and fault tolerance, making it a valuable tool for applications with demanding requirements.

Connecting to Redis Cluster

The Connection dialog now contains a Type drop-down where you can choose from a Standalone database instance or one which is part of a Cluster:

Selecting the Cluster item from the drop-down causes the Role drop-down to appear directly beneath it:

It allows you to choose between the Master database or a Replica (i.e. slave).

Other New Features In Navicat Premium 16.3

Navicat Premium 16.3 introduces a few other features, including support for the MongoDB Time-Series Collection as well as support for setting MySQL descending primary key.

New in version 5.0, the MongoDB Time-Series Collection efficiently stores sequences of measurements over a period of time. Time series data is any data that is collected over time and is uniquely identified by one or more unchanging parameters. The unchanging parameters that identify your time series data is generally your data source's metadata. Compared to normal collections, storing time series data in time series collections improves query efficiency and reduces the disk usage for time series data and secondary indexes.

Meanwhile, the MySQL descending primary key utilizes an index that stores rows in a descending order. The query optimizer will choose this type of an index when a descending order is requested by the query. This index type was introduced in MySQL 8.0.

Conclusion

In today's blog, we learned about some of the exciting new features in Navicat 16.3, namely support for Redis Cluster, MongoDB Time-Series Collections and MySQL descending primary keys.

Both Navicat Premium 16.3 and Navicat for Redis 16.3 are available for a free trial of 14 days on Windows, macOS and Linux.

Redis is a powerful open-source, in-memory data structure store that is used for various purposes such as caching, session management, real-time analytics, and more. One of the fundamental data types in Redis is strings, which can hold any kind of text or binary data, up to a maximum limit of 512 megabytes. In today's blog, we'll learn how to work with strings in Redis, both using the CLI and Navicat for Redis.

Using the Command Line Interface (CLI)

Redis provides a command-line interface (CLI) that allows users to interact with the database using a set of commands. Here's how you can work with strings via the Redis CLI:

1. Set a String

To set a string in Redis, you can use the `SET` command. This command assigns a value to a key.

    SET my_key "Hello, Redis!"

In this example, we're setting the value "Hello, Redis!" to the key `my_key`.

2. Get a String

To retrieve the value of a string, you can use the `GET` command.

    GET my_key

This command will return the value associated with the key `my_key`, which is "Hello, Redis!" in this case.

3. Append to a String

The `APPEND` command is used to append a value to an existing string. If the key doesn't exist, a new key is created with the provided value.

    APPEND my_key ", How are you?"

After this operation, the value of `my_key` will be "Hello, Redis!, How are you?".

4. Get a Substring

You can retrieve a substring from a string using the `GETRANGE` command. This command takes two arguments: the key and the range (start and end indices).

    GETRANGE my_key 0 4

Executing this command will return the substring "Hello" from `my_key`.

Using Navicat for Redis

Navicat for Redis is a powerful graphical user interface (GUI) tool that provides a user-friendly environment for working with Redis databases. Here's how you can perform string operations using Navicat:

1. Connecting to Redis

After launching Navicat, start by creating a new connection to your Redis server. Provide the necessary connection details such as host, port, and authentication credentials if required.

2. Navigating to the Keys

Once connected, you'll see the list of databases on the left-hand side. Expand the database containing the key you want to work with and navigate to the "Keys" section.

3. Setting a String

To set a string, right-click on the "Keys" section, select "Add Key", and choose "String" from the dropdown menu.

Enter the desired key name and value, then click "Apply". The new key will appear in the Keys List:

4. Getting a String

To retrieve the value of a string, simply double-click on the key in the "Keys" section. Navicat will display the key details, including its value.

5. Appending to a String

Right-click on the key and select "Edit Key" from the context menu. You can then append the desired text to the existing value.

Conclusion

This blog covered how to work with strings in Redis, both using the CLI and Navicat for Redis. Working with strings in Redis is a fundamental aspect of utilizing the database. Whether you choose to use the command-line interface or a GUI tool like Navicat for Redis, understanding how to set, get, append, and manipulate strings allows you to effectively manage your data.

In the world of relational databases, joining tables on foreign keys is a common and well-understood practice. However, there are situations where you need to join tables based on non-foreign key fields. This might seem unconventional, but it can be a powerful technique when used appropriately. In this article, we will explore the concept of joining database tables on non-foreign key fields, and we'll demonstrate how to do it using Navicat.

Why Join on Non-Foreign Key Fields?

In typical database design, tables are related using foreign keys, which establish clear relationships between data. However, there are scenarios where you might need to join tables based on fields that are not explicitly marked as foreign keys. Here are some reasons why you might consider this approach:

• Data Enrichment: You may want to enrich your data by combining information from different tables based on some shared characteristics.
• Legacy Databases: In legacy databases, foreign keys may not have been established, or the schema might not follow best practices.
• Data Migration: During data migration or integration, you might need to join data from multiple sources.
• Complex Queries: Some complex analytical or reporting queries may require joining tables on non-foreign key fields.

Using Navicat for Non-Foreign Key Joins

Navicat is a powerful database client that supports various database management systems like MySQL, PostgreSQL, SQL Server, and more. It provides a user-friendly interface for designing queries, making it an excellent choice for joining tables on non-foreign key fields.

Example: Combining Customer and Order Data

Let's consider a scenario where you have two tables: Customers and Orders. Normally, these tables would be related through a CustomerID foreign key field in the Orders table. However, in this example, we want to join them based on a shared Email field, which is not a foreign key.

To join the Customers and Orders tables on the Email field, you can use a SQL query like this:

SELECT Customers.*, Orders.*FROM CustomersINNER JOIN Orders ON Customers.Email = Orders.CustomerEmail;

In this query:

• Customers.* and Orders.* select all columns from both tables.
• INNER JOIN combines rows with matching Email and CustomerEmail values.

Tips for Non-Foreign Key Joins

When joining tables on non-foreign key fields, consider the following tips:

• Data Consistency: Ensure that the non-foreign key fields you're joining on have consistent data. In our example, the Email field should be consistently formatted and not contain missing or duplicate values.
• Indexes: Consider creating indexes on the fields you're joining on. Indexes can significantly improve query performance.
• Data Types: Ensure that the data types of the fields being joined match. For example, if you're joining on an email address, both fields should have the same data type, such as VARCHAR.
• Testing: Always thoroughly test your queries to verify that the results are as expected, especially when joining tables on non-foreign key fields.

Conclusion

Joining database tables on non-foreign key fields is a flexible and powerful technique that can help you work with data in unconventional ways. Navicat provides an intuitive interface for crafting SQL queries that perform these joins, making it a valuable tool for database professionals and developers.

Remember that while joining on non-foreign key fields can be useful, it should be done thoughtfully and with attention to data quality and consistency. When used appropriately, this approach can unlock new insights and possibilities in your data analysis and reporting.

Redis is a high-performance, in-memory data store known for its speed and versatility. One of its useful features is the ability to set expiration times for keys. Expiring keys in Redis is crucial for managing data and ensuring that outdated or temporary data is automatically removed from the database. In this article, we'll explore how to expire keys in Redis using the redis-cli and Navicat for Redis as well as how this feature can be applied in various scenarios.

Setting Expiration for a Key

To set an expiration time for a key in Redis, you can use the EXPIRE or SETEX command. The EXPIRE command allows you to set an expiration time in seconds, while SETEX sets both the key's value and its expiration time in one command. Here's how to use both commands:

Using EXPIRE:

127.0.0.1:6379> SET mykey "Hello, Redis"OK127.0.0.1:6379> EXPIRE mykey 60(integer) 1

In this example, we first set the value of mykey to "Hello, Redis" using the SET command. Then, we use EXPIRE to set an expiration time of 60 seconds for mykey. After 60 seconds, the key will automatically be removed from the database.

Using SETEX:

127.0.0.1:6379> SETEX mykey 60 "Hello, Redis"OK

With SETEX, we achieve the same result in a single command by specifying the key, the expiration time (60 seconds in this case), and the value.

Checking the Time-to-Live (TTL)

To check the remaining time until a key expires, you can use the TTL command. This command returns the remaining time in seconds or -2 if the key does not exist or -1 if the key exists but has no associated expiration time (it will never expire). Here's how to use it:

127.0.0.1:6379> TTL mykey(integer) 30

In this example, we check the remaining time for mykey, which was set to expire after 60 seconds. The command returns 30, indicating that there are 30 seconds left until the key expires.

Removing Expired Keys

Redis automatically removes keys when their expiration time is reached. However, you can also manually delete keys using the DEL command. This can be useful if you want to remove a key before it expires. Here's how to use it:

127.0.0.1:6379> DEL mykey(integer) 1

In this example, we use the DEL command to remove mykey manually. After running this command, the key will no longer exist in the database.

Setting Key Expiration in Navicat

In Navicat, the data editor includes a TTL drop-down for setting a key's expiration:

Options include "No TTL", "Expire In (seconds)" and "Expire At (Local Time)". Here's how to expire a key in 60 second:

The key's expiry information will be set when the Apply button is clicked.

Common Use Cases for Expiring Keys

Expire keys in Redis can be used in various scenarios to manage data effectively:

1. Caching

Redis is often used as a caching layer. By setting short expiration times for cache keys, you can ensure that the cache remains fresh and relevant, preventing the storage of stale data.

2. Session Management

Managing user sessions in a web application becomes more manageable with Redis. Setting session data to expire after a certain period of inactivity can help free up resources and enhance security.

3. Rate Limiting

Rate limiting is a common use case for API throttling. Redis can be used to count and limit the number of requests from a client within a specific time frame by expiring rate limit keys after a predefined time.

4. Temporary Data Storage

Redis can serve as a temporary data store for background jobs or temporary data processing. Expiring keys automatically clean up data that is no longer needed, reducing manual intervention.

Conclusion

In this article, we learned how to expire keys in Redis using the redis-cli and Navicat for Redis as well as how this feature can be applied in various scenarios. Key expiration in Redis is a powerful feature that helps manage data efficiently, ensuring that outdated or temporary data is automatically removed from the database. Whether you're using Redis for caching, session management, rate limiting, or temporary data storage, the ability to set expiration times for keys can significantly improve the performance and reliability of your applications.

In the realm of database management and development, the choice of tools can greatly impact efficiency and productivity. Java-based tools have emerged as strong contenders, offering diverse capabilities for working with databases. However, when it comes to native database connectivity, the differences between tools can be quite impactful. Let's explore how Navicat's ability to connect to databases natively sets it apart from other Java-based tools in the market.

Understanding Native Database Connectivity

Native database connectivity refers to a tool's ability to communicate directly with a database using the database system's native protocol. This eliminates intermediaries or translation layers, resulting in optimized and efficient connections. Java-based tools that support native database connectivity can harness the inherent optimizations and features provided by each database system, leading to enhanced performance and smoother workflows.

The Efficiency Factor

Navicat's standout feature is its native database connectivity, which significantly enhances efficiency. When compared to some Java-based tools, the difference becomes evident. Native connectivity eliminates the need for additional translations, resulting in faster data transfer, query execution, and overall performance. This can be crucial for managing large datasets, executing complex queries, and ensuring real-time interactions with the database.

Streamlined Development Workflows

Java-based tools that lack native database connectivity might encounter bottlenecks in development workflows. These tools often require additional steps for data translation and interpretation, leading to delays in coding and testing. Navicat's native connectivity streamlines development by directly communicating with the database system, reducing wait times, and enabling agile iterations. This agility is a boon for developers seeking optimal productivity.

Accuracy in Data Manipulation

Another area where Navicat's native connectivity shines is data manipulation. Java-based tools that rely on intermediate layers might introduce inaccuracies during data transformation and visualization. Navicat's direct interaction with the database's native format ensures accurate data previews, making it an ideal choice for tasks involving data analysis, transformation, and reporting.

Security and Compatibility

Native connectivity not only enhances efficiency but also contributes to security and compatibility. Java-based tools may require additional configurations to match the authentication and authorization mechanisms of different database systems. Navicat's native connectivity adheres to these protocols, providing enhanced security and better compatibility with the latest features and updates of supported databases.

Final Thoughts

When comparing Navicat's ability to connect to databases natively with other Java-based tools, it's clear that the former offers distinct advantages. Native database connectivity propels Navicat's efficiency, development workflows, accuracy in data manipulation, and compatibility. These benefits collectively contribute to a seamless database management and development experience.

As the field of database management continues to evolve, the importance of native connectivity becomes even more pronounced. By choosing a tool like Navicat, which prioritizes native connectivity, users can harness the full potential of their database systems, optimize their workflows, and ensure secure and reliable interactions with their data.

In Redis, a Hash is a data structure that maps a string key with field-value pairs. Thus, Hashes are useful for representing basic objects and for storing groupings of counters, among other things. This article will go over some of the main commands for managing hashes, both via the redis-cli and using Navicat for Redis.

Creating and Updating a Hash

In Redis, the key is the name of the Hash and the value represents a sequence of field-name field-value entries. For instance, we could describe a vehicle object as follows:

vehicle make Toyota model Crown trim Platinum year 2023 color black

To work with Hashes, we use commands that are similar to what we use with strings, since Hash field values are strings. Case in point, the command HSET sets field in the Hash to value. If key does not exist, a new key storing a hash is created. If field already exists in the hash, it is overwritten.

HSET key field value

For each HSET command, Redis replies with an integer as follows:

• 1 if field is a new field in the hash and value was set.
• 0 if field already exists in the hash and the value was updated.

Let's create the vehicle hash described above:

HSET vehicle make "Toyota"   // 1HSET vehicle model "Crown"   // 1HSET vehicle trim "Platinum" // 1HSET vehicle year 2015       // 1HSET vehicle color "black"   // 1

Now, if we update the value of the year field to 2022, HSET returns 0:

HSET vehicle year 2022 // 0

Creating a Hash in Navicat

In the Navicat for Redis, Hash fields may be added using the built-in Editor. Clicking on the ellipsis [...] button on the right of a field opens a special Editor where you can enter individual field values:

Clicking the Apply button adds the new Hash or field.

Fetching a Hash Field's Value

We can fetch the value associated with field in a Hash using the HGET command:

HGET key field

For example, we can use it to verify that we are getting 2022 as the value of year instead of 2015:

HGET vehicle year // 2022

We can also get all hash contents (fields and values) using the HGETALL command:

HGETALL key

Let's try it out:

HGETALL vehicle/* Returns:makeToyotamodelCrowntrimPlatinumyear2022colorblack*/

HGETALL replies with an empty list when the provided key argument doesn't exist.

Deleting a Field

The HDEL command removes the specified fields from the hash stored at key. Specified fields that do not exist within this hash are ignored. HDEL returns the number of fields that were removed from the hash. If a key does not exist, it is treated as an empty hash and HDEL returns 0.

HDEL key field [field ...]

Let's use HDEL to delete the year and color fields:

HDEL vehicle year color // 2

In the Navicat Editor, we can remove a field by selecting it and clicking the Delete [-] button located under the fields list:

Conclusion

This blog article highlighted some of the main commands for managing Hashes in Redis, both via the redis-cli and using Navicat for Redis.

Interested in giving Navicat for Redis a try. Download it here. The trial version is fully functional for 14 days.

Redis HashHash redis-cli Navicat for Redis Hash

Hash

Redis key Hash value

vehicle make Toyota model Crown trim Platinum year 2023 color black

Hash string Hash HSET Hash Hash Hash

HSET key field value

HSET Redis

• Hash 1
• Hash 0

vehicleHash

HSET vehicle make "Toyota"   // 1HSET vehicle model "Crown"   // 1HSET vehicle trim "Platinum" // 1HSET vehicle year 2015       // 1HSET vehicle color "black"   // 1

year 2022HSET 0

HSET vehicle year 2022 // 0

Navicat Hash

Navicat for Redis Hash [...]

Hash

Hash

HGET Hash

HGET key field

year 20222015

HGET vehicle year // 2022

HGETALL Hash

HGETALL key

HGETALL vehicle/* Returns:makeToyotamodelCrowntrimPlatinumyear2022colorblack*/

HGETALL

HDEL Hash Hash HDEL Hash HashHDEL 0

HDEL key field [field ...]

HDEL year color

HDEL vehicle year color // 2

Navicat [-]

redis-cli Navicat for Redis Redis Hash

Navicat for Redis 14

Redis supports several data types for storing collections of items. These include lists, sets, and hashes. Last week's blog article covered the List data type and highlighted some of the main commands for managing them. In today's follow-up we'll be turning our attention to the set type. In Redis, a Set is similar to a List except that it doesn't keep any specific order for its elements and each element must be unique. This article will go over some of the main commands for managing sets, both via the redis-cli and using Navicat for Redis.

Creating a Set

In Redis, we can create a Set by using the SADD command that adds the specified members to the key:

SADD key member [member ...]

As mentioned previously, each element must be unique. For that reason, specified members that are already part of the Set are ignored. If the key doesn't exist, a new Set is created and the unique specified members are added. If the key already exists or it is not a Set, an error is returned.

Here's the command to create a "vehicles" set:

SADD vehicles "Infiniti"         // 1SADD vehicles "Mazda"            // 1SADD vehicles "Ford" "Mercedes"    // 2SADD vehicles "Porsche" "Mercedes" // 1

Note that the SADD command returns the number of members that were added in that statement, not the size of the Set. We can see in the last line that only one element was added as there was already a "Mercedes" value.

Creating a Set in Navicat

In the Navicat for Redis Editor, Set values are represented as a Elements. Clicking on the ellipsis [...] button on the right of the Element opens a special Editor where you can enter individual Set elements:

Clicking the Apply button adds the new Set or element. Navicat automatically removes duplicate values.

Removing Members From a Set

We can remove members from a Set by using the SREM command:

SREM key member [member ...]

SREM vehicles "Mazda" "Mercedes" // 2SREM vehicles "Dodge" // 0

Similar to the SADD command, SREM return the number of members that were removed.

In the Navicat Editor, we can remove any Set element by selecting it and clicking the Delete [-] button located under the Element values:

Verifying That a Value Exists

To verify that a member is part of a Set, we can use the SISMEMBER command:

SISMEMBER key member

If the member is part of the Set, this command returns 1; otherwise, it returns 0:

SISMEMBER vehicles "Infiniti" // 1SISMEMBER vehicles "Alfa Romeo" // 0

Viewing a Set

To show all the members that exist in a Set, we can use the SMEMBERS command:

SMEMBERS key

Let's see what is currently contained in the vehicles Set:

SMEMBERS vehicles// returns "Infiniti", "Ford", "Porsche"

Merging Sets

We can combine Sets very easily using the SUNION command:

SUNION key [key ...]

Each argument to SUNION represents a Set that we want to merge into a larger Set. Note that any overlapping members will be removed in order to maintain element uniqueness.

Say that we had another Set named more_vehicles that contained the values "Corvette" and "Alfa Romeo". We could view all the members of both the vehicles and more_vehicles Sets as follows:

SUNION vehicles more_vehicles// "Infiniti", "Ford", "Porsche", "Corvette", "Alfa Romeo"

Conclusion

This blog article highlighted some of the main commands for managing Sets in Redis, both via the redis-cli and using Navicat for Redis.

Interested in giving Navicat for Redis a try. Download it here. The trial version is fully functional for 14 days.

In a resounding testament to its commitment to excellence and innovation, Navicat has been announced as the winner of the prestigious Best DBA Solution category in the 2023 DBTA Readers' Choice Awards. (Navicat Data Modeler was also a finalist in the Best Data Modeling Solution category.) The annual awards program, hosted by Database Trends and Applications (DBTA) magazine, celebrates outstanding products and solutions in the dynamic landscape of data management and analytics. The recognition is a result of votes and opinions from DBTA's readershipcomprising data and IT professionals hailing from diverse industries.

Comprising the perfect mix of cutting-edge technology and pragmatic utility, Navicat has solidified its place as a leading provider of natively designed database management and development solutions. The award underscores Navicat's dedication to empowering data professionals with tools that streamline database operations and enhance development endeavors.

The award-winning category, Best DBA Solution, stands as a nod to Navicat's unrivaled expertise in this realm. With Navicat's flagship product, Navicat Premium, data professionals are equipped with an all-in-one database management tool that supports a multitude of database systems, including MySQL, PostgreSQL, MariaDB, MongoDB, SQL Server, Oracle, SQLite, and now Redis. The tool's hallmark user-friendly interface combined with powerful features like data visualization, data modeling, and data synchronization have secured Navicat's spot as a trusted solution for over 50% of Fortune 500 companies, effectively meeting their diverse database management needs.

Ken Lin, the CEO of Navicat, expressed his elation and honor at receiving this prestigious recognition. "We are thrilled and honored to receive this recognition from the DBTA readers," Lin remarked. "At Navicat, we are committed to providing our customers with the best possible tools for managing and developing their databases, and this award is a testament to the hard work and dedication of our team."

One of Navicat's notable features lies in its platform-specific design approach. The tools are natively designed for specific platforms, ensuring a seamless and optimized experience that aligns naturally with the system in use. This approach offers stability, usability, and an intuitive experience for efficient database management.

Navicat's commitment to continuous innovation and improvement is evident through its product range, culminating in the recent release of Navicat 16.2, which introduces Redis compatibility an enhancement poised to further elevate users' capabilities in the database management sphere.

As the data management landscape continues to evolve, Navicat's commitment to innovation and excellence positions it at the forefront of the industry, a trusted partner for professionals navigating the complexities of database management and development.

In this recent blog article, we learned about Redis' six data types. Redis Lists contain a collection of strings, sorted by the same order in which they were added. This article will expand on the List data type and highlight some of the main commands for managing them.

List Performance

In Redis, it's important to note that Lists are implemented as linked lists. A linked list is one whose nodes contain a data field as well as a "next" reference (link) to the next node in the list:

This has some important implications regarding performance. It is fast to add elements to the head and tail of the List but it's slower to search for elements within the List as we do not have indexed access to the elements (like we do in an array).

Creating a List

A List is created by using a Redis command that pushes data followed by a key name. There are two commands that we can use: RPUSH and LPUSH. If the key doesn't exist, these commands will return a new List with the passed arguments as elements. If the key already exists or it is not a List, an error is returned.

RPUSH

RPUSH inserts a new element at the end of the List (at the tail):

RPUSH key value [value ...]

Let's create a "guitars" key that represents a List of guitar brands:

RPUSH guitars "Jackson" // 1RPUSH guitars "Fender"  // 2RPUSH guitars "Gibson"  // 3

Each time we insert an element, Redis replies with the length of the List after that insertion. After the above three statements, the guitars should contain the following three elements:

Jackson Fender Gibson

LPUSH

LPUSH behaves the same as RPUSH except that it inserts the element at the front of the List (at the header):

LPUSH key value [value ...]

We can use LPUSH to insert a new value at the front of the guitars list as follows:

LPUSH guitars "Ibanez" //4

We now have four guitars, starting with "Ibanez":

Ibanez Jackson Fender Gibson

Creating a List in Navicat

In the Navicat for Redis Editor, list values are represented as Elements. Clicking on the ellipsis [...] button on the right of the Element opens a special Editor where you can enter individual list elements:

Clicking the Apply button adds the new list or element.

Once added, an element's position in the list may be changed using the up and down arrow buttons.

Fetching List Items using LRANGE

LRANGE returns a subset of the List based on a specified start and stop index:

LRANGE key start stop

We can see the full List by supplying 0 and -1 for the start and stop indexes:

LRANGE guitars 0 -1 //returns Ibanez Jackson Fender Gibson

Meanwhile, the following command retrieves the first two guitars:

LRANGE guitars 0 1 //returns Ibanez Jackson

Removing Elements from a List

LPOP removes and returns the first element of the List while RPOP removes and returns the last element of the List. Here are some examples:

LPOP guitars //returns Ibanez RPOP guitars //returns Gibson 

In the Navicat Editor, we can remove any List element by selecting it and clicking the Delete [-] button located under the Element values:

Conclusion

This blog article highlighted some of the main commands for managing Lists in Redis, both via the redis-cli and using Navicat for Redis.

Interested in giving Navicat for Redis a try. Download it here. The trial version is fully functional for 14 days.

Since Redis is a key-value store that lets us associate values with a key, it does not use the Data Manipulation Language (DML) and query syntax as relational databases. So how to we write, read, update, and delete data in Redis? This tutorial will cover how to write, read, update, and delete keys using the redis-cli as well as Navicat for Redis.

Reading Data

We can use the GET command to ask Redis for the string value of a key:

GET key

Here's an example in Navicat for Redis that fetches the value for the key "auth service" shown below:

As expected, it returns its value of "auth0":

However, if we try to fetch the value for "indiana_jones_episodes", we get an error "WRONGTYPE Operation against a key holding the wrong kind of value". That's because its value is a zset. Since Redis supports 6 data types, you need to know what type of value that a key maps to, as for each data type, the command to retrieve it is different.

Here are the commands to retrieve key value(s):

• if value is of type string -> GET <key>
• if value is of type hash -> HGET or HMGET or HGETALL <key>
• if value is of type lists -> lrange <key> <start> <end>
• if value is of type sets -> smembers <key>
• if value is of type sorted sets -> ZRANGEBYSCORE <key> <min> <max>
• if value is of type stream -> xread count <count> streams <key> <ID>.

So, to retrieve the values for "indiana_jones_episodes", we can use ZRANGEBYSCORE and include the min and max arguments as follows:

That returns the first three values of the sorted set.

Writing and Updating Data

In Redis, the SET key Value command works for both setting the initial value as well as for updates.

Of course, in Navicat, both keys and values can be modified at any time using the Editor:

Deleting Data

In Redis, we can use the DEL command to delete a key, along with all of its associated values. It's syntax is:

DEL key

For example, the following command would delete the "auth service" key:

DEL "auth service"

Be careful; Redis does not ask you to confirm the operation!

In Navicat, we can delete a key by selecting it in the table and then clicking the Delete [-] button. A dialog will ask us to confirm before proceeding with the delete, in case we happened to click it by accident!

Conclusion

In this tutorial, we learned how to write, read, update, and delete keys using the redis-cli as well as Navicat for Redis. Next time, we'll learn some more commands for working with data using redis-cli commands, along with how to accomplish the same thing using Navicat.

Interested in giving Navicat for Redis a try. Download it here. The trial version is fully functional for 14 days.

More than a mere database, Redis can also act as a message broker that supports both publishing and subscribing (pub/sub) operations. This blog will provide a brief overview of Redis's Pub/Sub capabilities using Navicat for Redis.

About the Publish/Subscribe Messaging Paradigm

Pub/Sub is a pattern whereby a sender (broadcaster) cannot send messages to specific receivers (subscribers). Instead, published messages are sent over channels, without any knowledge of how many (if any) subscribers are tuning in. Subscribers then sign up for one or more channels so that they only receive messages that are of interest to them. Decoupling publishers and subscribers in this way allows for greater scalability and makes it easier to manage the flow of information in a complex system.

Redis Pub/Sub provides a lightweight, fast, and scalable messaging solution that can be used for various purposes, such as real-time notifications, sending messages between microservices, or communicating between different parts of a single application.

Message Delivery in Redis

Redis employs an at-most-once message delivery system. As the name suggests, it means that a message will be delivered only once, if at all. As such, once the message is sent by the Redis server, it's never sent again. If the subscriber is unable to receive the message (for example, due to an error or a network outage) the message is forever lost. Much like catching your favorite show on the radio, if you happen to miss it, you're out of luck. If your application requires stronger delivery guarantees, you should use Redis Streams instead.

Moreover, Pub/Sub has no relation to the key space. This means that a message published on database 10 will be heard by a subscriber on database 1. If you need scoping, Redis suggests prefixing the channel name (i.e., prod_mychannel, test_mychannel).

Publishing with Navicat for Redis

In Navicat for Redis (or Navicat Premium), we can access the Pub/Sub screen via the Pub/Sub button on the main toolbar. From there, we can publish messages using the Publish Pane:

In Redis, channels are not explicitly created by the user. The channels are created automatically when either the first message is published or a client subscribes to them. To demonstrate, we'll open two connections with the same Redis server. Each connection will act as a different client. The first connection will subscribe to the "test_channel", while the second one will publish a message to the same channel. By doing so, we would expect our message to be delivered to the subscriber as soon as it's published.

To subscribe to a channel in Navicat, we simply need to click the Subscribe button. That will open the Subscribe Dialog:

There, we would enter the channel name - "test_channel" - and then click Subscribe. After the dialog closes, the channel will appear in the Channels list, along with a record of the subscribe action:

To publish a message in Navicat, we would select it in the Channels list (it is the default since we only have one channel at this point), enter out message in the Message text field, and click on Publish. At that point, we should see a notification that the message was received:

Conclusion

In this blog, we explored Redis's Pub/Sub capabilities using Navicat for Redis.

Interested in giving Navicat for Redis a try. Download it here. The trial version is fully functional for 14 days.

SQL supports the use of aliases to give a table or a column a temporary name. Not only can they save on typing, but aliases can also make your queries more readable and understandable. In today's blog, we'll learn how to incorporate aliases into our queries using Navicat Premium 16.2.

Overview of SQL Aliases

As mentioned in the introduction, both table and column names may be aliased. Here is the syntax for each:

Alias Column Syntax

SELECT   column_name [AS] alias_name,  column_name AS 'Alias Name' -- for names with spacesFROM table_name;

Alias Table Syntax

SELECT column_name(s)FROM table_name [AS] alias_name;

Two points to consider regarding aliases:

• An alias is usually preceded by the AS keyword, but it is optional.
• An alias only exists for the duration of that query.

Table Aliases in Join Queries

Here's a query against the Sakila Sample Database that fetches information about all copies of a particular film:

SELECT *FROM film f   INNER JOIN inventory i ON i.film_id = f.film_idWHERE i.store_id = 1 AND f.title = "Academy Dinosaur";

In the above query, since both the film and inventory tables contain a film_id column, they must be fully qualified, i.e., prefixed by the table name. In this case, aliases may be employed to shorten the statement.

Here is the query in Navicat along with the results:

Column Aliases

In the case of column names, abbreviations are often utilized to keep column names short when designing database tables. For example:

• "so_no" for "sales order number".
• "qty" stands "quantity"

Here, column aliases may be employed to make the column contents more intuitive. Here's an example:

SELECTinv_no AS invoice_no,amount,due_date AS 'Due date',cust_no 'Customer No'FROMinvoices;

You can also assign column aliases to expressions, as seen below:

The above query selects both the current and future price of products after applying a price increase.

Limitations of Column Aliases

Since column aliases are assigned in the SELECT clause, you can only reference the aliases in the clauses that are evaluated after the SELECT clause. Hence, you cannot include aliases in the WHERE clause; doing so will result in an error:

This happens because the database evaluates the WHERE clause before the SELECT clause. Therefore, at the time it evaluates the WHERE clause, the database doesn't have the information of the NewPrice column alias.

It is however permissible to use column aliases in the ORDER BY clause because it is evaluated after the SELECT clause:

The database evaluates the clauses of the query in the following order:

FROM > SELECT > ORDER BY

Table Aliases and Navicat

In Navicat, once a table alias has been defined, it will come up in the auto-complete list.

That makes using aliases even more time saving!

Final Thoughts on Using Database Aliases

In today's blog, we learned how to incorporate aliases into our queries using Navicat Premium 16.2. Aliases are an easy way to make your queries more readable and understandable, which is important because code isn't just about execution; it's also a communication mechanism.

One of the key features that sets Redis apart from other key-value stores is its support of numerous data types, which include strings, lists, sets, sorted sets, and hashes. This makes it easier for developers to solve problems because they tend to know which data type to use for a given task. This blog will outline the six data types supported by Redis.

Strings

Redis stores strings as a sequence of bytes. Strings in Redis are binary safe, meaning they have a fixed length rather than have it determined by one or more special terminating characters. As such, you can store anything up to 512 megabytes in one string.

In Navicat for Redis, we can create a new key/value pair via Edit -> Add Key from the main menu. That will add a new empty row in the Data View and open the Editor:

We can use the Editor to set the:

• Key Name
• Key Type
• Value
• TTL (Time To Live)

Clicking the Apply button populates the new row with the contents of the Editor form. It also shows the size of the value:

Hashes

In Redis, a hash is a collection of key value pairs. As such, they are a good choice for representing objects and to store groupings of counters, among other things. Every hash can store up to 2^32 - 1 field-value pairs (that's more than 4 billion!).

For hash values, the Navicat Editor employs a table with Field and Value columns:

Lists

Redis Lists are simply lists of strings, sorted by the same order in which they were added. You can add elements to a Redis List on the head or on the tail. The max length of a list is 2^32 - 1, or 4294967295, elements (that's more than 4 billion of elements per list!).

In the Navicat Editor, list values are represented as an Element. Clicking on the ellipsis [...] button on the right of the Element opens a special Editor where you can enter the complete list:

Sets

Redis Sets are an unordered collection of strings. A Set is similar to a list, except that Set doesn't allow duplicates and doesn't preserve insertion order.

Sets can be sorted as well. In a Sorted Set, every member is associated with a score, that is used in order to take the sorted set ordered, from the smallest to the greatest score. While members remain unique, the scores may be repeated.

Navicat handles Sets much in the same way as Lists. Here's an example:

Sorted Sets are listed as "zset" in the Key Type drop-down:

Streams

The Redis stream data type was introduced in Redis 5.0. Streams model a log data structure but also implement several operations to overcome some of the limits of a typical append-only log.

Yes, Navicat for Redis supports the Stream data type!

Final Thoughts on Redis Data Types

This blog outlined the six data types supported by Redis, including the new Stream type.

Interested in giving Navicat for Redis a try. Download it here. The trial version is fully functional for 14 days.

Redis is an open source, BSD licensed, advanced key-value store, written in C. It's also referred to as a data structure server, since the keys can contain strings, hashes, lists, sets and sorted sets. This tutorial will provide the fundamentals of Redis concepts needed to start using it right away.

Why Use Redis?

Redis is certainly not the only key-value store to choose from. However, it does offer some advantages over its competitors. For instance:

• Redis supports more data types than most other key-value data stores. Developers already know most types, including list, set, sorted set, and hashes. This makes it easy to solve problems since developers tend to know which data type to use for a task.
• Redis holds its database entirely in memory, using the disk only for persistence, making it exceptionally fast. In fact, it can perform about 110,000 SETs per second, about 81,000 GETs per second!
• Redis can replicate data to any number of slaves.
• All Redis operations are atomic, which ensures that if two clients concurrently access the same data, Redis server will receive the updated value(s).
• Redis natively supports Publish/Subscribe, making it ideal for messaging-queues.
• Redis is well suited for managing any short-lived data in your application, such as web application sessions, web page hit counts, etc.

When Not to Use Redis

Of course, Redis is not without its flaws. It's not your best choice if you need to minimize the chance of data loss in case of outages, such as a sudden loss of power. You can configure multiple save points, such as every five minutes and/or 100 writes against the data set. However should Redis stop working without a proper shutdown for any reason, you should be prepared to lose the latest several minutes of data.

Another issue is that Redis often needs to fork a child process in order to persist data to disk. This can consume a lot of system resources if the dataset is large, and may result in an interruption of service for clients ranging from a few milliseconds to a full second, depending on dataset size and CPU power.

Installing Redis

How you install Redis depends on your operating system and whether you'd like to install it bundled with Redis Stack and Redis UI. The official Redis site has guides for every O/S:

• Install Redis on Linux
• Install Redis on macOS
• Install Redis on Windows
• Install Redis with Redis Stack and RedisInsight
• Install Redis from Source (requires C compiler and libc)

Once you have Redis up and running, and can connect using Navicat for Redis, and continue with the tutorial below.

Exploring the Redis CLI

Navicat for Redis includes a console, which allows you to communicate directly with a database instance:

One advantage to using the CLI in Navicat is that it provides auto-completion on every aspect of CLI commands, including command names as well as their parameters:

Conclusion

This tutorial provided the fundamentals of Redis concepts needed to start using it right away. There will be plenty more articles on Redis in the coming weeks, so be sure to check back often!

Version 16.2 of Navicat Premium added several exciting new features to an already stellar product, the most noteworthy being Redis support. Now, there is a Navicat administration and development client specifically for Redis. Navicat for Redis offers users an easy-to-access visual interface to visualize and optimize Redis data. It includes a rich set of features for making routine management tasks simpler, easier and more efficient than ever before. It can connect to any local/remote Redis server, and is compatible with cloud databases like Redis Enterprise Cloud, Amazon ElastiCache, Google Memorystore and Microsoft Azure. This blog will outline some of Navicat for Redis's most important features.

Data Viewer

Beyond a simple key/value store, Redis is actually a data structures server with support for many different kinds of values. Viewing such complex data structures can be a challenge, but not with Navicat for Redis's data viewer; you can use it to view, edit, search and sort keys and data via the classic spreadsheet-like Grid View, Tree View, all with a built-in editor. Navicat provides you with the tools you need to manage your data in both a smooth and efficient manner.

Data view in Windows:

Query Editing

As with other Navicat products, Navicat for Redis lets you create, edit and run queries within the Query Editor, all without having to worry about syntax and proper usage of commands. It helps you to code quickly thanks to Code Completion, which gives you suggestions for keywords and reduces the repetition in coding.

Query Editor:

You can also save queries and commands as Snippets that you can reuse over and over again.

Pub/Sub

The Pub/Sub Tool allows you to send messages and subscribe to specific channels using a simple and intuitive UI. You can save the channels as a profile, or assign colors to the channels to easily distinguish the corresponding channels and their messages. You can choose from active channels, custom channels, and custom patterns.

Pub/Sub Tool:

Collaboration

Navicat for Redis includes Navicat Cloud, which allows you to synchronize your connection settings, queries, snippets and virtual group information to the cloud. The Navicat Cloud service provides real-time access to these, and share them with your coworkers anytime and anywhere.

Navicat Cloud:

Dark Mode

The dark theme helps protect your eyes from the brightness of the standard "white" computer themes. Most importantly, dark mode only affects how application screens look, and does not alter their behavior in any way.

Redis data in dark mode

User Management

Users and their associated permissions can be managed using an intuitive interface. There, you can create, edit and delete users in minutes without having to type commands, as well as easily create new privilege groups to apply multiple sets of rules to a user all at once.

User management screen:

Conclusion

This blog outlined just some of Navicat for Redis's most important features. There are plenty of others, including Backup/Restore facilities, Automation, Secure Connections, and more.Cross-platform licensing is now available. Whether you're a Windows, macOS, or Linux user, you can purchase once and select a platform to activate and transfer your license at a later date if needed.

During the process of normalization, groups of fields that represent a distinct entity are removed from a larger and/or more central table to a separate one. Common fields (usually IDs) are then employed to maintain their relationship. We can see an example below:

In relational database, referential integrity between tables is enforced using foreign key constraints.

This blog will cover how foreign keys work as well as how to create a foreign key constraint in MySQL using Navicat 16 for MySQL .

About the Film and Inventory Relation

The model that we saw in the intro depicts a one-to-many relationship between the film and inventory tables whereby a film entity (1 row) may link to zero or more entities (rows) in the inventory table.

The film table is called the parent table or referenced table, and the inventory table is known as the child table or referencing table. As such, the foreign key columns of the child table often refer to the primary key columns of the parent table.

In this example, we are only focusing on one relation. In fact, a table can have more than one foreign key where each foreign key references to a primary key of the different parent tables.

Once a foreign key constraint is in place, the foreign key columns from the child table must have the corresponding row in the parent key columns of the parent table or values in these foreign key columns must be NULL. For example, each row in the inventory table has a film_id that exists in the film_id column of the film table. Multiple rows in the inventory table can have the same film_id.

In the next section we'll create a Foreign Key Constraint for this relationship in Navicat 16 for MySQL.

Creating a Foreign Key Constraint in Navicat

In Navicat, you'll find Foreign Key Constraints on the Foreign Keys tab of the Table Designer. To create a new Foreign Key Constraint, open the parent table - in this case film - in the Table Designer and click the Add Foreign Key button. That will create a new row in the Foreign Keys list:

Next, select the "film" table from the Fields drop-down, the "inventory" table from the Referenced Table drop-down and the "film_id" for the Referenced Fields:

The next step is to choose the On Delete and On Update actions. MySQL supports five different referential options, as follows:

• CASCADE: It is used when we delete or update any row from the parent table, the values of the matching rows in the child table will be deleted or updated automatically.
• SET NULL: It is used when we delete or update any row from the parent table, the values of the foreign key columns in the child table are set to NULL.
• RESTRICT: It is used when we delete or update any row from the parent table that has a matching row in the reference(child) table, MySQL does not allow to delete or update rows in the parent table.
• NO ACTION: It is similar to RESTRICT. But it has one difference that it checks referential integrity after trying to modify the table.
• SET DEFAULT: The MySQL parser recognizes this action. However, the InnoDB and NDB tables both rejected this action.

Let's follow the example of the existing FK and choose an On Delete action of RESTRICT and an On Update action of CASCADE:

Finally, click the Save button to create the new Foreign Key Constraint. Note that Navicat will create the name for you if you do not populate the Name field.

Conclusion

Foreign Keys play an essential role in maintain referential integrity between tables. As such, one should be created for every table relationship. Navicat 16 for MySQL makes it quite easy to manage your Foreign Key Constraints without having to write any SQL commands.

As part of the process of normalizing database tables, redundant columns are extracted from higher level tables into separate subsidiary ones. This often occurs due to some fields having a one to many relationship with the parent entity. For example, take the following model that was generated using Navicat Data Modeler:

Appraisals were initially part of the ups table, but this led to data redundancy because there can be multiple vehicles appraised in one visit. Therefore, it made sense to remove the vehicle fields from the ups table and place them in their own table.

The drawback to normalization to third normal form (3NF) is that you wind up with a lot of ID fields in the main table. As a database practitioner looking at a table, it becomes very challenging to know what entity each ID column points to. As an illustration, take a look at the ups table from the above model diagram, and notice how the CSRs, customers, and vehicles have all been reduced to numeric IDs that don't help identify the underlying entities in any way:

This is partially related to the use of auto-incrementing IDs as well as normalization, but, in any event, we can make the data much easier to read by creating a view. A database view is a subset of a database and is based on a query that runs on one or more database tables. Database views are saved in the database as named queries and can be used to save frequently used, complex queries.

In Navicat 16, we can create a new view by choosing File -> New -> View... from the main menu:

That will add a new view tab.

The next step is to add the SQL statement that will generate the view fields:

If you need any help in writing your statement, there are Preview, Explain, View Builder, and Beautify SQL buttons on the tab toolbar.

Let's say we don't want to wait for the view to be created before viewing the results, we can click the Preview button to see it now:

Now the ID columns contain more descriptive - and meaningful - textual data .

Under the tab buttons, there are three more tabs - Definition, Advanced, and SQL Preview. The Advanced tab contains additional options such as the Algorithm, Definer, Security, and Check option, while the SQL Preview shows the generated CREATE VIEW statement:

The new view is named `Untitled` until we save it. At that point, a dialog appears in which we can specify the View Name:

Upon saving, the new view will be added to the Navigation Pane on the left-hand side and may be summoned at any time:

Final Thoughts on Creating Views in Navicat 16

In today's blog, we learned about database views and went through the process of making one to help identify records in a table that links to a number of dependent tables via ID fields.

Whereas most database systems employ locks for concurrency control, PostgreSQL does things a little differently: it maintains data consistency by using a multi-version model, otherwise known as Multi-Version Concurrency Control, or MVCC for short. As a result, when querying a database, each transaction sees a snapshot of data as it was some time before, regardless of the current state of the underlying data. This prevents the transaction from viewing inconsistent data that could be caused by other concurrent transaction updates on the same data, and provides transaction isolation for each database session. This blog article will provide a brief overview of how the MVCC protocol works as well as cover some of the pros and cons of the MVCC approach.

The MVCC Protocol Explained

The main difference between lock models and MVCC is that the latter ensures that reading never blocks writing and writing never blocks reading.

In MVCC, every transaction has a transaction-timestamp that indicates (a) when it started and (b) when a transaction updates a certain data-item, such as a field, or a record, or a table. A new version of this data-item is created while also retaining the older version. Each version is provided with:

• a write-timestamp to indicate the timestamp of the transaction (i.e. the time the transaction started) that created it and
• a read-timestamp to indicate the latest timestamp of all the transactions that have read it.

The basic idea of the MVCC protocol is that the transaction manager only allows operations if they can be allowed in a manner that is consistent with all transactions executing in their entirety at the moment of their timestamp. This is referred to as the presumed execution order. Database researcher Jan Hidders explains how the transaction manager accomplishes this as follows:

If a transaction wants to read an item, it is given access to the version that it would have read in the presumed execution order. This will be the one with the latest write-timestamp just before its own timestamp. For example, if there are versions with write-timestamps 5, 12 and 20, and the timestamp of the transaction is 14, then the version with write-timestamp 12 is the one read by this transaction in the presumed execution order.

If a transaction wants to write an item, it is checked if there is not a read operation that was allowed earlier and that in the presumed execution order would read the new version caused by the requested write operation, but when it was allowed read another version. For example, assume again we have versions with write-timestamps 5, 10 and 16. Moreover assume the read-timestamps of these versions are 8, 12 and 20, respectively. If a transaction with timestamp 11 wants to update the item, there is a problem, because the version with write-timestamp 10 was read by a transaction with timestamp 12. So, if a version with timestamp 11 is created, the transaction with timestamp 12 would in the presumed execution order not have seen the version created by the transaction with timestamp 10, but the one that is now about to be created with timestamp 11. If, on other hand, a transaction with timestamp 14 wants to write the item, this is fine, since as far as we know after t=12 in the presumed execution order the item was not read by any transaction until the moment it was updated at t=16.

Pros and Cons of MVCC

Pros:

• As you can tell from the description above, all read operations will always be allowed immediately. This is usually not the case in a lock-based approach, where read-locks might be refused because of existing write-locks.
• It tends to allow also more write operations to go through immediately than lock-based approaches usually do.

Cons

• If a write operation is refused, there is no alternative but rolling back or restarting the transaction: once the update is refused it will also be refused if we retry it later. This differs from lock-based approaches, where we usually can wait until the lock becomes available. It is for this reason that MVCC is categorised as an optimistic protocol: it is very efficient if there are no conflicts, but once there is one you may have to undo a lot of work.
• The many versions of an item might require significantly more storage space. In lock-based approaches only the one version needs to be stored.
• The removal of versions that are no longer needed can cause some overhead.

Final Thoughts on Multi-Version Concurrency Control in PostgreSQL

This blog article provided an overview of how the MVCC protocol works and presented a few of its pros and cons.

Interested in working with PostgreSQL? You can try Navicat 16 for PostgreSQL for FREE for 14 days!

At the top of Navicat Monitor 3's Query Analyzer screen, there's a chart that shows queries with the longest wait times:

It's essential to identify laggard queries because they can bring everything crashing to a crawl.

Besides fixing a slow query once it's been identified, another strategy might include limiting query execution times across the board. In professional grade databases such as PostgreSQL, there are settings to cap query execution time for the entire database or even per user, via the statement_timeout variable. In this blog, we'll learn how to work with this important database variable in Navicat 16 For PostgreSQL.

Setting the statement_timeout Variable at the Database Level

Setting a default statement timeout for your database is an excellent starting point. This ensures that any application or person connecting to the database will not have queries running longer than that. A sane default would be either 30 or 60 seconds, but you can go higher if you wish. Here a statement that sets a value of 60 seconds:

ALTER DATABASE mydatabase SET statement_timeout = '60s';

In Navicat 16 For PostgreSQL we can view the statement_timeout via Tools > Server Monitor > PostgreSQL from the main menu. You'll find it on the Variables tab:

In fact, you may want to employ the Find tool in order to pinpoint the statement_timeout variable, as there are many! You can click the Highlight All toggle button to better help identify the variable once matched.

Of course the Show statement works as well:

Setting a Query Timeout for a Specific User

For even more fine grained control, we can set a query timeout value for a specific user (you know, the one who always selects the entire database!). This is achieved using the ALTER ROLE statement, which can set many database variables, including statement_timeout.

To try it out, let's create a new user role called "guest":

Now we can use the ALTER ROLE statement to limit query execution time as follows:

ALTER ROLE guest SET statement_timeout='5min';

We can query the pg_roles table to obtain information about the statement_timeout (including how it was set):

The rolconfig value is an array, so we can unnest it to get one setting per row:

Final Thoughts on Setting Query Timeouts in PostgreSQL

It's crucial to be able to identify laggard queries because they can bring your database performance down to a crawl. For that, there's Navicat Monitor 3's Long Running Queries chart at the top of the Query Analyzer screen.

Another approach is to limit how long a query can execute before it times out. As we saw in today's blog, in PostgreSQL, this can be done at the database, session, and even at the individual role level.

If you haven't already setup your statement_timeout variable(s), I would encourage you to do so ASAP. This is just one component of proper database tuning that will help to ensure your database instance stays healthy and available.

Interested in giving Navicat 16 For PostgreSQL a try? You can download the fully functioning application here to get a free 14 day trial!

The idea behind database auditing is to know who accessed your database tables and when, along with what modifications were made to them. It's not only considered to be the standard minimum requirement for any enterprise level application, but is also a legal requirement for many domains such as banking and cybersecurity. Database Audit Trails are essential in investigating all sorts of application issues such as unauthorized access, problematic configuration changes, and many more.

In today's blog, we're going to add logging to the MySQL Sakila Sample Database to audit the rental table. It's a key table because the database represents the business processes of a DVD rental store.

Creating a Table to Store Audit Trail Data

Ideally it's best to have an audit table for each table being audited. Here's the DDL statement to create the audit trail table for the rental table:

create table rental_audit_log( id int NOT NULL AUTO_INCREMENT, rental_id int NOT NULL, old_values varchar(255), new_values varchar(255), done_by varchar(255) NOT NULL, done_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY (id));

Alternatively, in Navicat, you can use the Table Designer to specify all of the fields and attributes without having to write a DDL statement:

Creating the Audit Logging Triggers

We'll need to create 3 database triggers to insert records in the rental_audit_log table, one for each type of DML statement performed on the rental table (INSERT, UPDATE, and DELETE).

AFTER INSERT Trigger

INSERT statements on the rental table will be intercepted by the rental_insert_audit_trigger. We'll get it to fire AFTER Insert operations and provide all of the new data as a JSON_OBJECT. In Navicat, all of those details may be supplied on the Triggers tab of the Table Designer:

After adding a new row to the rental table, we can see a new record in the rental_audit_log as well:

AFTER UPDATE Trigger

UPDATE statements on the rental table will be captured by the following rental_update_audit_trigger:

Now, every time a rental record is updated, the rental_update_audit_trigger is executed, and a rental_audit_log row will be created to capture both the old and the new state of the modified record. In this case, we can see that user robg changed the rental_date from "2005-05-25 17:17:04" to "2005-05-31 19:47:04":

AFTER DELETE Trigger

To track DELETE statements on the rental table, we will create the following rental_delete_audit_trigger:

In this case, only the old_values column is set since there is no new record state. Hence the empty new_values column in the generated rental_audit_log row:

Here, we can see that user fsmith deleted record 1114 from the rental table on 2023-03-22 at 08:46:07.

Final Thoughts on Audit Trail Logging Using Triggers

In today's blog, we added logging to the MySQL Sakila Sample Database to audit the rental table. Our logging table included some of the most common audit fields. Some organizations include others, such as the DML operation type, while others only include changed fields. It's really whatever works best for the organization.

Once again, Navicat has been nominated for the prestigious Database Trends and Applications (DBTA) Readers Choice Awards in the following categories:

• Best DBA Solution (Navicat)
• Best Data Modeling Solution (Navicat Data Modeler)
• Best Database Development Solution (Navicat for MySQL)
• Best Database Performance Solution (Navicat Monitor)

Right now is the time to vote for Navicat as the voting period will only be open until Wednesday May 10, 2023.

Winners will be showcased in a special section on the DBTA website and in the August 2023 issue of Database Trends and Applications magazine.

More on Navicat and Nominated Products

Navicat is owned by PremiumSoft. Founded in 1999, PremiumSoft has developed a wide variety of applications for Windows, macOS, Linux and iOS. Navicat is the choice of over 5 million database users all around the world. Over 180,000 registered customers across 7 continents and 138 countries have chosen our products. More than 50% of the Fortune 500 rely on Navicat every day. Some notable customers include Apple Inc., Google Inc., Oracle, Intel, Microsoft, Fujitsu, Accenture, HP, IBM, Ebay, Samsung, Sony, JP Morgan, KPMG, Barclays, DHL, Federal Express, General Electric, and many more.

Navicat Data Modeler 3 is a powerful and cost-effective database design tool which helps you build high-quality conceptual, logical and physical data models. It supports various database systems, including MySQL, MariaDB, Oracle, SQL Server, PostgreSQL, and SQLite.

Navicat 16 for MySQL is the ideal solution for MySQL/MariaDB administration and development.

Navicat Monitor 3 is a safe, simple and agentless remote server monitoring tool that is packed with powerful features to make your monitoring effective as possible. Monitored servers include MySQL, MariaDB, PostgreSQL and SQL Server.

Winner in 2022!

All of the votes from satisfied Navicat customers helped Navicat take the BEST DATABASE DEVELOPMENT SOLUTION category for Navicat for MySQL. In doing so, Navicat beat Quest Toad for Oracle and Devart dbForge Studio.

Navicat Premium also was a finalist for the BEST DBA SOLUTION award, along with Devart dbForge Studio.

About Database Trends and Applications

Database Trends and Applications is a magazine that covers data and information management, big data, and data science. In addition, their website, dbta.com, provides white papers, webinars, and other offerings for learning in the field. DBTA also circulates newsletters that connect subscribers with news and analysis about a diverse range of subjects such as Oracle News, Linux News, MultiValue News, General Information Management News, and more.

Where to Vote

You can cast your vote for your favorite Navicat product directly on the DBTA website. To make your selections:

• Locate the desired category on the page, i.e., Best DBA Solution and select Navicat from the list of candidates:

  

• Once you've made your selections for the categories which you are voting for, enter your name, company, and email at the bottom of the page in the Complete Your Vote section and click the Submit button:

  

You should see the form replaced with a "Thanks for voting!" message confirming a successful form submission.

Now it's Your Turn to Vote!

Don't forget to cast your votes by Wednesday May 10, 2023. Again, the categories and nominated Navicat products are:

• Best DBA Solution (Navicat)
• Best Data Modeling Solution (Navicat Data Modeler)
• Best Database Development Solution (Navicat for MySQL)
• Best Database Performance Solution (Navicat Monitor)

A table column, such as one that stores first names, may contain many duplicate values. If you're interested in listing the different (distinct) values, there needs to be a way to do so without resorting to complex SQL statements. In ANSI SQL compliant databases like PostgreSQL, SQL Server, and MySQL, the way to select only the distinct values from a column is to use the SQL DISTINCT clause. It removes duplicates from the result set of a SELECT statement, leaving only unique values. In this blog article, we'll learn how to use it.

Syntax and Behavior

To use the SQL DISTINCT clause, all you need to do is insert the DISTINCT keyword between in SELECT and columns and/or expressions list like so:

SELECT DISTINCT columns/expressionsFROM tables[WHERE conditions];

You may include one or more columns and/or expressions in your statement, as the query uses the combination of values in all specified columns in the SELECT list to evaluate their uniqueness. Also, if you apply the DISTINCT clause to a column that has NULL values, the DISTINCT clause will keep only one NULL and eliminate the others. In other words, the DISTINCT clause treats all NULL values as the same value.

One Column Example

A common use case for a query is to list all of the cities and/or countries of an organization's customers or users. Here's a query in Navicat Premium 16 against the classicmodels sample database:

As highlighted with the red outline, there are duplicate cities.

To get a list of unique cities, we can add the DISTINCT keyword to the SELECT statement:

We can utilize Navicat's code-completion feature to bring up the DISTINCT keyword. Navicat displays information in drop-down lists as you type your SQL statement in the editor, it assists you with statement completion and the available properties of database objects, for example databases, tables, fields, views etc with their appropriate icons:

Multiple Column Example

The DISTINCT keyword may also be applied to multiple columns. In that context, the query will only return rows where all of the selected columns are unique. First, let's add the country field to our last query:

Once again, we see duplicates, which makes sense because a duplicated city will likely reside in the same country.

Once again, adding the DISTINCT keyword will cause the query engine to look at the combination of values in both city and country columns to evaluate and remove the duplicates:

DISTINCT with Null Values

As mentioned above, the DISTINCT clause treats all NULL values as the same value so that only one instance of NULL is included in the result set. We can test that out for ourselves by querying a column such as this on in the same customers table that we queried previously:

As predicted, adding the DISTINCT keyword removed all but one instance of NULLs :

Final Thoughts on Selecting Distinct Values From a Relational Database

In this blog article, we learned how to use the SQL DISTINCT clause, which removes duplicates from the result set of a SELECT statement, leaving only unique values. As we saw, it can work on one or more columns as well as NULL values. However, should you need apply an aggregate function on one or more columns, you should use the GROUP BY clause instead.

Stored Procedures, Functions, and Views

Welcome to the 3rd and final installment on SQL naming conventions. In Part 1, we covered the rules for naming tables, while Part 2 explored conventions for column names. This installment will offer some guidelines for naming other database objects such as Stored Procedures, Functions, and Views.

Stored procedures

A stored procedure is a set of statement(s) that perform some defined actions. Typically, they contain statements that are used frequently. Stored procedures are similar to functions in programming in that they can accept parameters, and perform operations when we call them.

General Format

Most DBAs like to give their stored procedures a prefix that identifies it as such, followed by the action that the stored procedure takes and then the name representing the object or objects it will affect:

[prefix]_[action]_[object]

Actions that you may take with a stored procedure include:

• Insert
• Delete
• Update
• Select
• Get
• Validate

Choosing a Prefix

The most obvious prefix to use on a stored procedure is "sp_". That being said, there's at least one good reason to avoid it as it's already used by SQL Server as a standard naming convention in the master database. If you do not specify the database where the object is, SQL Server will first search the master database to see if the object exists there and then it will search the user database. Even if you don't host your database(s) on SQL Server, you should probably avoid using this as a naming convention, just in case you ever switch.

Instead, consider a prefix like "usp_" instead.

Putting It All Together

Here are a few examples of well-named stored procedures to formulate your own:

• usp_insert_person
• usp_delete_person
• usp_update_person
• usp_select_person
• usp_get_person
• usp_validate_person

User-defined Functions

Similar to built-in database functions, a user-defined function accepts only input parameters and contains a set of SQL statements that perform actions and return the result, which can be either a single value or a table.

The naming convention a for user-defined function is to have an "fn_" prefix, followed by its action. Hence, the syntax should be very similar to that of stored procedures:

[prefix]_[action]_[object]

Functions that return true or false may follow the rule of using "is" or "are" as the action (verb).

Some examples of function names would include:

• fn_count_string_instances
• fn_get_customer_balance
• fn_is_inventory_in_stock
• fn_get_column_type

Views

A view is a "virtual table" in a database that is defined by a query. A view can combine data from two or more table, using joins, and also just contain a subset of information. This makes them convenient to abstract, or hide, complicated queries.

The naming conventions for a view should have a "v_" or "vw_" prefix, followed by the namespace, results. As such, the syntax should be:

[prefix]_[result]

Here are a few examples:

• vw_actor_full_name
• vw_sales_by_store
• v_staff_list
• v_sales_by_product_category

Final Thoughts on Naming Conventions for Stored Procedures, Functions, and Views

In this three part series, we explored some commonly used naming conventions and considered how best to formulate our own. Part 1 covered Table names, while Part 2 focused on column names. Finally, Part 3 addressed Naming Conventions for other database objects such as Procedures, Functions, and Views.

Remember that you need not apply rules to all database objects. You could choose to apply naming convention rules to tables and column names only. It's really your decision, as using a naming convention is not mandatory, but beneficial nonetheless.

Column Names

Welcome to the 2nd installment on SQL naming conventions. As mentioned in part 1, naming conventions are a set of rules (written or unwritten) that should be utilized in order to increase the readability of the data model. These may be applied to just about anything inside the database, including tables, columns, primary and foreign keys, stored procedures, functions, views, etc. Having covered the rules for naming tables in part 1, we'll be looking at column names in this installment. Other database objects such as Procedures, Functions, and Views will be explored in part 3.

The Primary Key Column

The primary key is a field or a combination of fields in a table that uniquely identify the records in the table. A table can have only one primary key. As such, many DBAs prefer to simply name this column "id". Other's append the "_id" suffix to the table name, as seen here in the Sakila Sample Database:

Likewise, you should also assign your PK constraint a meaningful name. The naming convention for a primary key constraint is that it should have a "pk_" prefix, followed by the table name, i.e. "pk_<table_name>".

Foreign Key Columns

A foreign key is a field in the table that references a primary key in other tables. A good rule to follow is to use the referenced table name and "_id", e.g. customer_id, employee_id. This will help us identify the field as a foreign key column and also point us to the referenced table.

Here's a city table that contains a foreign key to the country table's country_id field:

The naming convention for a foreign key constraint is to have an "fk_" prefix, followed by the target table name, followed by the source table name. Hence, the syntax should be "fk_<target_table>_<source_table>".

Following the foreign key constraint naming convention for the city table would give us the name "fk_city_country":

Data Columns

In the section on Describing Real-World Entities in Part 1, it states:

Any time that you're naming entities that represent real-world things, you should use their proper nouns. These would apply to tables like employee, customer, city, country, etc. Usually, a single word should exactly describes what is in that table.

The same rules can and should be applied to data columns. Again, you should use the least possible words to describe what is stored in that column, e.g., country_name, country_code, customer_name. If two tables will have columns with the same name, you could add something to keep the name unique, although that's not strictly necessary as table prefixing will differentiate the columns in queries. Nonetheless, having unique names for each column is helpful because it reduces the chance to later mix these two columns while writing queries. Names like customer_name city_name are likely to come up in more than one table. If that concerns you, you can always make the names more descriptive, such as order_customer_name or city_of_residence_name.

Dates

For dates, it's good practice to describe what the date represents. Names like start_date and end_date are pretty common and generic. You can describe them more precisely by using names like call_start_date and call_end_date.

Final Thoughts on Naming Conventions for Column Names

You probably noticed from all of the examples presented that both table and column names should be in lowercase with words separated by an underscore ("_"). For example, customer_name and invoice_date as opposed to customerName and invoiceDate. This works well with the SQL style convention of capitalizing statements names, clauses, and other keywords for better code readability, e.g. SELECT customer_name, invoice_date FROM orders;

Navicat Monitor 3 added support for PostgreSQL, one of the most popular modern relational databases in use today. New features include an SQL Profiler for PostgreSQL instances as well as enhanced Query Analyzer and Long Running Queries pages, both of which were touched upon in the Monitoring PostgreSQL with Navicat Monitor 3.0 blog article. Today's topic will be the Instance Details page.

Instance Details Page at a Glance

The Instance Details page is accessible by clicking on an instance card in the Overview page. It shows server parameters and metrics in a visual way, providing you with a quick view of the server load and performance. There, you can view:

• the top 5 databases based on size
• the top 5 tables based on size
• system charts on CPU, Memory, Swap, and DB Disk Usage
• metrics on connections, queries, tables, buffer, cache and sort, as well as locks

Information on the Instance Details Page

The Instance Details page is split up into the following three sections:

• Summary
• Databases & Tables
• Charts

Here's a breakdown of each section:

Summary

The summary displays host information about the server, server properties, alerts and status. There, you can view or edit the instance variables as well as click on raised alerts to open the alert page.

Databases & Tables

This section displays the top five databases and tables by size, as well as a sixth category called "Others" that groups the remaining databases or tables. You can hover over each segment to show the size percentage. To view size information of all databases and tables in the instance, click the View All button.

Charts

Navicat Monitor displays visualizations of server performance metrics as small charts. The charts track and refresh the data at certain intervals, which are configurable by the user. Related metrics are displayed using different predefined colors and symbols. Note that, due to size constraints, the axis scales and labels of the small charts are not printed.

Both the time interval (X-axis) and refresh options are configurable via the AUTO REFRESH drop-down menu, the START FROM datetime picker, the time INTERVAL drop-down menu and the panning arrows:

Hovering the mouse pointer over a chart will display the values at that point:

You can also click on a chart to open the Chart page in order to view the details of an individual chart or to see more charts.

Pausing (and Resuming) Monitoring

You can pause and resume monitoring the instance here by using the Pause Monitoring and Resume Monitoring buttons, located in upper-right quadrant of the Instance Details page:

Navicat Monitor stops collecting information from the server until the monitoring resumes and adds the "[Paused]" indicator beside the names of paused instances in the Instances Pane (on the left):

Final Thoughts on Viewing PostgreSQL Instance Details in Navicat Monitor 3

Today's covered Navicat Monitor 3's Instance Details screen. It shows the server parameters and metrics in a highly visual way, gives you a quick view of the server load and performance, and more!

Navicat Monitor 3 comes packed with a variety of exciting new features. Case in point, you can now create traces that collect query data based on selected filters from the server log. When creating a trace, you can define criteria to filter the data collected by SQL Profiler and set a schedule for executing the trace. In today's blog we'll learn how to create a trace and view its results.

More about Traces

The Tracing feature is part of the SQL Profiler, which is only available for PostgreSQL. The SQL Profiler provides graphical query execution details for locating inefficient and slow queries.

The data collected from traces may be analyzed and used to troubleshoot performance issues. For example, you can see which queries are affecting performances in the production environment.

Creating a Trace

You can create new traces on the SQL Profiler, Query Analyzer, and Long Running Queries pages by clicking the Add Trace icon or + New Trace.

On the SQL Profiler page you'll have to select the instance before clicking the + New Trace button:

Clicking the Add Trace icon or + New Trace button will bring up the New Trace dialog. A prompt may pop up asking you to authorize Navicat Monitor to get relevant data from your instance.

Here are all of the details that you can enter on the New Trace dialog:

• TRACE NAME: the name of the trace.
• USER FILTER: the users/roles whose queries to include in the trace. Empty means including queries from all users/roles.
• DATABASE FILTER: the databases to trace. Empty means including queries against all databases.
• QUERY FILTER: search strings or QueryIDs to filter queries for the trace.
• MAX TRACE ROW COUNT: the maximum number of rows for the trace. SQL Profiler will terminate the trace when it reaches the row count.
• SCHEDULE: scheduling details for executing the trace.
• Share with: who can see the trace.

Here's the New Trace dialog with some of the above fields filled in:

Clicking on Create Trace that starts tracing according to the provided schedule. You should then see results after the first time period has elapsed.

Viewing Trace Results

A trace provides a graphical representation of the execution plan for each query with statistics for its components. Here is the trace for a query against the Sakila Sample Database:

You can see from the above screen capture that the Trace Results are divided into 3 sections:

• Query Table: The query table shows the basic information for the queries. Select a query to show its details and plans.
• Query Details: shows the complete statement of the query.
• Execution Plan: the execution plan that is generated for each query can be viewed in three different formats: Visual, Charts and Text-Based.

Final Thoughts on Tracing Queries on your PostgreSQL Instances with Navicat Monitor 3

In today's blog we saw how easy it is to create a trace and view its results in Navicat Monitor 3. Only available for PostgreSQL, traces collect query data based on selected filters from the server log. I think that you'll find them to be indispensable for locating inefficient and slow queries.

Version 3 of Navicat Monitor has just be released. Unsurprisingly, it packs many outstanding new features, as well as numerous improvements to existing features. One of the most noteworthy changes between version 2 and 3 is added support for PostgreSQL, including an SQL Profiler for PostgreSQL instances.

Today's blog will provide a quick guide on getting setup to monitor your be PostgreSQL instances using Navicat Monitor 3.0.

Adding a PostgreSQL Instance

You can see all of the monitored database instances on the Overview screen. In order to monitor our PostgreSQL instance, we need to add it to this screen. To do that, we simply need to click the "+New Instance" button at the top of the screen. Doing so presents a context list of available database types - both traditional and cloud-based:

Select the PostgreSQL item to open the New PostgreSQL Instance dialog:

Navicat Monitor can connect the database server over a secure SSH tunnel to send and receive monitoring data. It allows you to connect your servers even if remote connections are disabled or are blocked by firewalls.

In the PostgreSQL Server section, enter the following information:

• Host Name: The host name or IP address of the database server.
• Port: The TCP/IP port for connecting to the database server.
• Username: A monitoring user for connecting to the database server.
• Password: The login password of the monitoring user.
• Server Type: The type of the server. Can be Unix-like or Windows.

Navicat Monitor can also collect the DB server's system performance metrics such as CPU and memory resources. If you do not provide this login, you can still monitor your server, but system performance metrics will not be shown.

Once you've entered all of the above information, click the "New" button to create the new instance.

Activating a Token

Now that we've added our PostgreSQL instance, we're ready to activate it. To do that, we'll need to assign a token to it via Configurations > Activate Tolens & License Instances.

To activate the instance, we can locate it in the Unlicensed Instances list, check the box beside it, and click the License button to move it into the Licensed Instances list. Here's our "PostgreSQL Test DB 1" instance in the Licensed list:

We can now receive server statistic details about our instance's performance regarding query execution as well as server load, availability, disk usage, network I/O, table locks and more. By easily tracking the deviations and traffic among servers, we can examine possible solutions and adjust our server settings accordingly.

Monitoring Query Performance

The Query Analyzer tool provides a graphical representation of the query logs that makes interpreting their contents much easier. In addition, the Query Analyzer tool enables us to monitor and optimize query performance, visualize query activity statistics, analyze SQL statements, as well as quickly identify and resolve long running queries. Here's the Query Analyzer for our new instance:

There are no Long Running Queries at this time because the database is new and not currently in use.

Final Thoughts on Monitoring PostgreSQL with Navicat Monitor 3.0

Thanks to Navicat Monitor 3.0, we can now monitor our PostgreSQL instances, via many useful tools, including the Enhanced Query Analyzer and Long Running Queries screens.

Navicat Monitor 3.0 is available fore Windows, macOS (using Homebrew), and Linux. You can try Navicat Monitor 3.0 for 14 days free of charge to sample of all its new features before you buy.

It seems like only yesterday that Navicat Monitor 2.0 was released, adding great new features to an already stellar product. Now, version 3.0 is introducing yet more outstanding features, including:

• Support monitoring PostgreSQL instances.
• Support SQL Profiler for PostgreSQL instances.
• Enhanced Query Analyzer.
• Enhanced Long Running Queries.
• Many other new features and improvements.

Of course, all of Navicat Monitor's existing functionality remains in place, including:

• Agentless Architecture
• Real-time Performance Monitoring
• See how your instances are currently functioning easily
• Advanced root cause analysis
• Set custom alert thresholds
• Get notifications via email, SMS or SNMP
• Replication Monitoring
• Powerful Query Analyzer
• and more...

In today's blog, we'll be taking a look at the brand new Navicat Monitor 3.0 with the emphasis being on the new features listed above.

About Navicat Monitor

Navicat Monitor is a safe, simple and agentless remote server monitoring tool that includes many powerful features to make your monitoring effective as possible. You can access Navicat from anywhere via a web browser to access statistics on server load and performance regarding its availability, disk usage, network I/O, table locks and more. Using this valuable data, you can easily examine possible solutions, tune the databases, and address potential issues before they can become serious problems or costly outages.

Navicat Monitor is a server-based software which can be accessed from anywhere via a web browser. With web access, you can easily and seamlessly keep track of your servers around the world, around the clock.

Support for PostgreSQL Instances

Just as version 2 added support for SQL Server, version 3 can monitor PostgreSQL as well. Here are all supported DB types in the New Instance menu:

Selecting PostgreSQL from the list brings up the New PostgreSQL Instance dialog, which includes all of the configuration options that you might need for a PostgreSQL instance:

You'll be happy to know that the SQL Profiler also works for PostgreSQL instances. It's a tool that provides graphical query execution details for locating inefficient and slow queries. It supports the creation of traces to collect data about the queries executed on an instance. The data can later be analyzed and used to troubleshoot performance issues.

Enhanced Query Analyzer and Long Running Queries

The Query Analyzer tool provides a graphical representation of the query logs that makes interpreting their contents much easier. In addition, the Query Analyzer tool enables you to monitor and optimize query performance, visualize query activity statistics, analyze SQL statements, as well as quickly identify and resolve long running queries. In version 3, the Long Running Queries chart has been moved to the top of the page for easier identification:

Moreover, the entire Long Running Queries section provides the ability to drill down to specific intervals:

Information about Long Running Queries may be exported as a PDF or scheduled report.

Final Thoughts on Navicat Monitor 3.0

In this blog, we explored just a few of the exciting new features of Navicat Monitor 3.0, including Support for PostgreSQL Instances, as well as Enhanced Query Analyzer and Long Running Queries. There are plenty more new features that we'll be looking at over the next several weeks.

Navicat Monitor 3.0 is available for Windows, macOS (using Homebrew), and Linux. You can try Navicat Monitor 3.0 for 14 days free of charge to sample of all its new features before you buy.

Table Names

Naming conventions are a set of rules (written or unwritten) that should be utilized in order to increase the readability of the data model. You may apply these rules when naming anything inside the database, including tables, columns, primary and foreign keys, stored procedures, functions, views, etc. You need not apply rules to all database objects. For instance, it would be perfectly fine to limit naming convention rules to tables and column names. It's really your decision, as using a naming convention is not mandatory, but beneficial nonetheless. This three part series will present some commonly used naming conventions and provide some tips for formulating your own. Part 1 will cover Table names, while Part 2 will focus on column names. Finally, Part 3 will address Naming Conventions for other database objects such as Foreign Keys, Procedures, Functions, and Views.

Why You Should Use a Naming Convention

Databases rarely have a small number of tables. In fact, it's not at all uncommon to have hundreds of tables. By following a naming convention, you'll make your life a lot easier by increasing the overall model readability, and making it easier to locate database (DB) objects.

Another good reason is that the database will slowly evolve over time. Although changes to the schema are usually avoided and done only when necessary, changing the name of a database object could affect your application code in a myriad of ways. Since you can expect the database will remain, more or less, very similar to its initial incarnation, if you apply best practices from the start and continue using them as you add new objects, you'll keep your database structure well organized over time.

Singular vs. Plural Table Names

One of the most commonly asked questions regarding the naming of tables is whether to use the singular or plural form. There are many differing opinions on this matter. In fact, we can see both views expressed in the schemas of the MySQL classicmodels and sakila sample databases, with the former employing plural table names, and the latter utilizing singular naming:

If it helps, most DBAs go with singular names. One reason is that plural names like "users" "roles" could lead to some weird table names down the road such "users_have_roles" rather than "user_has_role".

Describing Real-World Entities

Any time that you're naming entities that represent real-world things, you should use their proper nouns. These would apply to tables like employee, customer, city, country, etc. Usually, a single word should exactly describes what is in that table.

There are times that you'll have to use more than one word to describe what is in a table. One such example can be seen in the classicmodels database. There is one table for "orders" and another for "order_details":

The "orders" table contains fields such as the Customer ID, Employee ID, Order Date, Shipped Date, Freight, and Shipping Address. Meanwhile, the "order_details" contains data about the products ordered, such as the Quantity ordered and Price. The field could have been named "product_details" but that would not convey that the product was associated with an order.

Naming Related Tables

For relations between two tables, it's standard practice to use both tables' names. A verb may also be added between both names to describe what that action is, for example "user_has_role", or simply "user_role". The Sakila Sample Database follows this convention by joining related tables with an intermediary table that combines both names. We can observe two examples in the database model below - "film_actor" and "film_category":

Final Thoughts on Table Naming Conventions in SQL

Don't be afraid to stray from a naming convention if it doesn't make logical sense in a given situation. For example, if we had a product and invoice table, and we wanted to specify which products were on which invoice, the name "invoice_item" might make more sense than either "invoice_product" or "invoice_contains_product".

It's official; the age of Artificial Intelligence (AI) has arrived! Until our new overlords decide to use us to power their machines, let's take the time to fully enjoy all the benefits they provide and the myriad of ways that they make our lives easier. Case in point, the AI-driven chatbot, ChatGPT, by OpenAI, has been lauded for its ability to produce tremendously spot-on answers to questions across a broad range of topics. And, although ChatGPT may not be making our jobs obsolete just yet, it has proven to be amazingly adept at working with data sets, much like a DBMS. In today's blog, we'll explore how ChatGPT could be utilized to supplement a professional database development and administration tool like Navicat.

Creating the Data Set

ChatGPT is able to model a formal dataset from a list of delimited values. All you need to do is tell it what to do using regular, conversational language. ChatGPT is also able to answer followup questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests. We can see an example on the OUseful.Info blog that created a table named "racerresults". Here are the instructions given to ChatGPT, along with a sampling of the input data:

Treat the following as a tab separated dataset. Using just the first, third and fourth columns, treat the data as if it were a relational SQL database table called "racerresults" with columns "Race", "Driver" and "Team", and the "Race" column as a primary key column. Display a SQL statement that could create the corresponding table and populate it with the data.Bahrain20 Mar 2022Charles LeclercFERRARI571:37:33.584Saudi Arabia27 Mar 2022Max VerstappenRED BULL RACING RBPT501:24:19.293Australia10 Apr 2022Charles LeclercFERRARI581:27:46.548Emilia Romagna24 Apr 2022Max VerstappenRED BULL RACING RBPT631:32:07.986Miami08 May 2022Max VerstappenRED BULL RACING RBPT571:34:24.258Spain22 May 2022Max VerstappenRED BULL RACING RBPT661:37:20.475Monaco29 May 2022Sergio PerezRED BULL RACING RBPT641:56:30.265erc...

From the above instructions and data, ChatGPT generated the following CREATE TABLE and INSERT statements:

With the data in place, we're ready to run queries against it.

Querying a Data Set with ChatGPT

In terms of query formulation, ChatGPT shares some similarities with Navicat, in that both allow you to construct queries with little knowledge of SQL. To do that, Navicat features the Query Builder tool. Here it is in macOS:

As for ChatGPT, it takes a question phrased in regular, conversational language, and produces the required SQL statement(s).

This ability to translate natural prompts into structured outputs has made it appealing not only to database professionals but also to teams across a variety of industries. As per G2's guide on GitHub Copilot vs ChatGPT for coding, ChatGPT dominates computer software, IT services, marketing, financial services, and education management. This industry-wide adoption underscores its versatility, showing that its value extends far beyond fun use cases like data experiments or emoji assignments.

For instance, given the following list of historical figures:

We can simply as ChatGPT how it would query for the oldest historical figure. Here is the resulting SQL statement and explanation offered by ChatGPT:

Fun with Data

ChatGPT can do a lot more than generate queries; it can also think creatively to assign emojis to each historical figure:

Final Thoughts on Supercharging Your Queries with Navicat and ChatGPT

While AI bots like ChatGPT are a long way from replacing traditional database tools, they do offer another tool to database practitioners who are looking for new and innovative ways of approaching data-related tasks. At the time of this writing, ChatGPT was at capacity and unable to accept new users, but once things die down a bit, I would urge you to give ChatGPT a try.

Subqueries can be categorized into two types:

• A non-correlated (simple) subquery obtains its results independently of its containing (outer) statement.
• A correlated subquery references values from its outer query in order to execute.

When a non-correlated subquery executes (independently of the outer query), the subquery executes first, and then passes its results to the outer query. Meanwhile, a correlated subquery typically obtains values from its outer query before it executes. When the subquery returns, it passes its results to the outer query.

Now that we know the difference between a correlated subquery and its non-correlated counterpart, this blog will cover how to write a correlated subquery in Navicat Premium 16.

Syntax and Usage

A correlated subquery is evaluated once for each row processed by the parent statement. The parent statement can be a SELECT, UPDATE, or DELETE statement. Here's the syntax for a SELECT query:

SELECT column1, column2, ....FROM table1 outerWHERE column1 operator  (SELECT column1, column2   FROM table2   WHERE expr1 = outer.expr2);

A correlated subquery is one way of reading every row in a table and comparing values in each row against related data. It is used whenever a subquery must return a different result or set of results for each candidate row considered by the main query. In other words, you can use a correlated subquery to answer a multipart question whose answer depends on the value in each row processed by the parent statement.

A Practical Example

Here's a rather ingenious query from stackoverflow against the Sakila sample database that fetches the most viewed film per country.

The first step is to count how many times each film was viewed in each country. Here is the SELECT statement for that:

SELECT   F.title AS title,   CO.country_id AS country_id,  CO.country AS country_name,   count(F.film_id) as timesFROM customer C INNER JOIN address A ON C.address_id = A.address_idINNER JOIN city CI ON A.city_id = CI.city_idINNER JOIN country CO ON CI.country_id = CO.country_idINNER JOIN rental R ON C.customer_id = R.customer_idINNER JOIN inventory I ON R.inventory_id = I.inventory_idINNER JOIN film F ON I.film_id = F.film_idGROUP BY F.film_id, CO.country_id;

And here is the above query and results in Navicat Premium 16:

The next step is to convert the above results into a list of countries, along with the most viewed film title and the number of times it was viewed. Here's the full query with correlated subquery with an explanation to follow:

Explanation:

• Subquery: Fetches a list of movie count, grouped by country.
• GROUP_CONCAT(title ORDER BY times DESC SEPARATOR '|||') returns ALL titles in that 'row', with the most-viewed title first. The separator doesn't matter, as long as never occurs in a title.
• SUBSTRING_INDEX('...', '|||', 1) extracts the first part of the string until it finds "|||", in this case the first (and thus most-viewed) title.

Final Thoughts on Correlated Subqueries

In today's blog we learned how to write a correlated subquery using Navicat Premium 16. Be forewarned that correlated subqueries can be slow. However, with proper optimizing, their speed can be increased significantly.

As you are no doubt aware, updating text values in the database is a commonplace occurrence. Nonetheless, it is a rare database administrator (DBA) that doesn't feel some trepidation upon executing batch updates against production tables. In today's blog, we'll learn how to use the SQL REPLACE() function to replace either a complete or partial string in a table column.

A Typical Scenario

Here's a screenshot of the products table from the classicmodels sample database:

Suppose that the makers of Chef Anton products have decided to enclose their products in quotation marks (""). This would require a total of 4 steps:

1. Employ the LIKE operator to identify rows with Chef Anton products.
2. Parse out the product name.
3. Add the enclosing quotation marks.
4. Convert the SELECT QUERY to an UPDATE.

Let's go over each step.

Identify Rows with Chef Anton Products

As mentioned above, we can utilize the LIKE operator to identify rows with Chef Anton products. Each of these begins with the string "Chef Anton's ", so we can search for it. To do that, we will need to escape the single quote (') character and include the multi-character "%" wildcard. Here is the resulting query and results in Navicat Premium 16:

Parse Out the Product Name

The next step is to parse out the product name so that we can enclose it within quotation marks. To do that, we can employ the LEN() function to calculate the number of characters after the "Chef Anton's " portion of the string and supply that result to the RIGHT() function:

Add the Enclosing Quotation Marks

The last step in constructing the SELECT query is to add the quotes around the product name. Having parsed out the product name, we can provide it to the REPLACE() function as the first parameter, along with the concatenated (quoted) version as the 2nd parameter:

An alternative way to achieve the same end is to simply use the CONCAT() function and feed it each part of the string as follows:

SELECT CONCAT(         LEFT(ProductName, LENGTH('Chef Anton\'s ')),  '"', RIGHT(ProductName, LENGTH(ProductName)-LENGTH('Chef Anton\'s ')),  '"'       ) AS product_name FROM products WHERE ProductName LIKE 'Chef Anton\'s %'; 

Convert the SELECT QUERY to an UPDATE

All that's left to do now is to convert our SELECT query into an UPDATE. Having executed the query as a SELECT first, we can be confident that the UPDATE statement won't affect any other rows than the ones we're interested in. Here is the UPDATE query and results confirming that only two rows were updated:

Upon refreshing the products table, we can now see our updated values:

Final Thoughts on How to Perform a Search and Replace in SQL

In this blog, we learned how to update a string in a table column using a four step process. By building up the query as a series of SELECT statements, we can minimize the risk of inadvertently changing data that we did not intend to.

The Code Snippets feature was introduced to all "Non-Essentials" Navicat Database Administration and Development tools in version 12. Version 16 added Code Snippets to Navicat's cloud services so that users could save their Code Snippets to the cloud and share them across Navicat products. For those of you who are unfamiliar with the Code Snippets feature, it allows you to insert reusable code into your SQL statements when working in the SQL Editor. Besides gaining access to a collection of built-in snippets, you can also define your own snippets. We've talked about Code Snippets before. The March 14, 2018 blog, Using Navicat Code Snippets, provided a general overview of the Code Snippets feature. Today's blog will cover how to create your own custom Code Snippets. It's something that can make writing queries a whole lot easier!

Creating a Code Snippet from Scratch or from Selected Text

There are a couple of ways to create a brand new Code Snippet. The first is by clicking the Create Snippet button in the Code Snippet pane:

If you've got a particular piece of code that you'd like to save for future use, you can select it in the Query Editor, bring up the context menu (i.e. right-click in Windows), and select Create Snippet from the menu:

That will bring up the New Snippet dialog with the Code text field filled in for you:

From there you can choose a category from the Label drop-down. This will allow you to filter by type later when you need to bring it up. The pre-defined choices are "Comment", "DLL", and "Flow Control". If none of these apply, you can always add your own! The next time you access the New Snippet dialog, it will be included in the Label drop-down.

There's also a text area for Remarks in order to add some context to the Snippet.

Finally, don't forget to give your Code Snippet a name! Here's the New Snippet dialog with all of the fields filled in:

Once saved, the new Code Snippet will appear in the Code Snippet pane:

Setting Placeholders

Placeholders are tabbable text selections that act as arguments for Code Snippets. In fact, auto-completed function arguments are themselves always displayed as placeholders! For instance, here is the Count() function:

To create a placeholder in your Code Snippet, select the text that you'd like to be highlighted in the Code Snippet and click the Set selected text as placeholder button:

Placeholder text is highlighted by a colored box for easy identification:

You can remove a placeholder the same way using the Remove placeholder button.

Conclusion

In today's blog we learned how to create our own custom Code Snippets in any "Non-Essentials" edition of Navicat 16. It's something that can definitely make writing queries faster and easier!

Aliases temporarily rename a table or a column in such a way that does not affect the underlying table(s) or view(s). As a feature of SQL that is supported by most, if not all, relational database management systems, aliases are a great way to both simplify your queries and/or customize the column headers in your result sets. In this blog, we'll do both, using Navicat Premium 16.

Renaming Columns

A lot of database designers use abbreviations for the table column names to keep them short, for example:

• emp_no for "Employee Number"
• qty for "quantity"

The full value of abbreviated column names are not always intuitive to those who view the query results. To remedy that, you can use column aliases that give columns more descriptive names in the result set.

The syntax for column aliases is:

column_name [AS] alias_name

Note that the AS keyword is optional.

You can include whitespace in your aliases, by enclosing them within single (or double) quotes like this:

column_name AS 'Alias Name'

Here's an example query that includes a few column aliases:

Using Aliases for Expressions

You've probably noticed that, if a query contains expressions, the entire expression is utilized as the column header. For example:

Assigning a column alias to the expression makes it much more palatable:

Table Aliases

Table aliases follow the same rules as column aliases, but their purpose is different, as table aliases don't appear anywhere in the query results. In their case, the idea is to use a shorter name in order to associate columns with their table in a way that shortens your queries.

The basic syntax of a table alias is as follows:

SELECT column1, column2....FROM table_name [AS] alias_nameWHERE [condition];

A column that is associated to its table is called a qualified column name. Columns need to be qualified when two columns with the same name appear in the same SELECT statement. In fact, we saw an example of qualified column names in the column alias example above. Here's another query that contains two actor_id columns - one from the actors table and another from the film_actor table:

Although the above query is perfectly functional, we can shorten it by employing table aliases:

Notice that unambiguous columns, i.e., those which only appear in one table, do not need to be qualified.

Another way that table aliases are helpful comes into play when using modern database tools like Navicat. Thanks to the auto-suggest feature, typing the table alias causes a drop-down of suggestions. In the case of table aliases, the drop-down will contain all the table columns:

This greatly accelerates query writing, which is an important part of professional database development.

Conclusion

This blog provided an overview of column and table aliases, along with some practical examples in Navicat Premium 16.

If you're interested in learning more about Navicat Premium 16, you can try the full unrestricted version out for 14 days completely free of charge!

Tablespace Management

This 3rd and final part of the Navicat 16 and Tablespaces series will focus on how to manage tablespaces in MySQL using Navicat Premium 16. Recall that Part 1 presented some advantages offered by tablespaces, including Recoverability, Ease of Adding More Tables, Automatic Storage Management, and the Ability to Isolate Data in Buffer Pools for Improved Performance or Memory Utilization. The second installment provided more information on what tablespaces are, how they work and the types of default tablespaces you'll find in the various relational database products.

Creating a Tablespace

Just as Navicat provides Table and SQL Designers, there are also facilities for working with Tablespaces. To open the Tablespace Designer, click on Others -> Tablespace from the Main Toolbar:

In the Designer, click on the New Tablespace button in the Toolbar:

The fields shown in the Designer will depend on the type of database that your working with. In the case of MySQL, you'll see the following fields:

• Engine drop-down: For standard MySQL 5.7 releases, only the InnoDB engine supports tablespaces, so it's the only option in the drop-down. MySQL NDB Cluster 7.5 also supports tablespaces using the NDB storage engine.
• Path textbox: Specifies the path of the datafile/tempfile. Note that you have to include the ".ibd" file extension.
• Block Size drop-down: The block size for the tablespace. MySQL only supports a block size of 1024, or 1 MB, so be sure to select that option from the drop-down.
• Block Size Unit: The size of one data block. As mentioned above, MySQL only supports a block size of 1024, or 1 MB; for other database types, you may choose K, M, G, T, P or E to specify the size in kilobytes, megabytes, gigabytes, terabytes, petabytes, or exabytes.

You can see the generated SQL statement by clicking on the SQL Preview tab:

Navicat will issue the CREATE TABLESPACE statement upon clicking the Save button. Here are the New Tablespace form fields after a successful Save operation:

Before saving the tablespace, Navicat will present a dialog for entering the tablespace name that will be utilized to display the tablespace in the Tablespaces Objects list:

Hence, entering a name of "classicmodels" will add it as seen below:

Altering a Tablespace

Selecting a tablespace from the Tablespaces Objects list will enable the Design Tablespace button in the toolbar for editing. If the database in question does not allow tablespace editing, as in the case of MySQL, the form fields will be disabled:

Otherwise, data can be modified and re-saved.

Deleting a Tablespace

Selecting a tablespace from the Tablespaces Objects list also enables the Delete Tablespace button in the toolbar. Clicking it will bring up a confirmation dialog that requires the user to check a box indicating that the Delete action is permanent and cannot be undone:

The user may also click the Cancel button to close the dialog without deleting the tablespace.

Conclusion of Navicat 16 and Tablespaces Series

Tablespaces allow database administrators to better control the physical storage layout by putting some tables on faster or more redundant disks, or to stripe tables across disks. This series covered both the theoretical side as well as more practical matters of tablespace management, from their creation to deletion using Navicat Premium 16.

How They Work

"What is it? It's it" - Epic, Faith No More

Welcome back to this series on working with tablespaces in Navicat 16. Part 1 presented some advantages offered by tablespaces, including Recoverability, Ease of Adding More Tables, Automatic Storage Management, and the Ability to Isolate Data in Buffer Pools for Improved Performance or Memory Utilization. This second instalment will provide more information on what tablespaces are, how they work and the types of default tablespaces you'll find in the various relational database products. The next and final part of the series will focus on how to manage tablespaces in Navicat 16.

Tablespaces As Containers

You can think of tablespaces as containers. These can be a directory name, a device name, or a file name. A single tablespace can have several containers. And, although it is possible for multiple containers (from one or more tablespaces) to be created on the same physical storage device, you will get the best performance if each container you create utilizes a different storage device. The figure below illustrates the relationship between tables and tablespaces within a database:

Tablespaces and the Database Manager

The database manager's role is to balance the data load across containers. As a result, all containers are used to store data to a lesser of greater degree. At the same time, the database manager does not always start storing table data in the first container. The number of pages that the database manager writes to a container before using a different container is called the "extent size".

The figure below shows the components of a tablespace, including the extent size:

Default Tablespaces

Most relational databases come with their own built-in tablespaces. Here are a few examples:

Oracle

Oracle comes with the following default tablespaces: SYSTEM, SYSAUX, USERS, UNDOTBS1, and TEMP:

• The SYSTEM and SYSAUX tablespaces store system-generated objects such as data dictionary tables. You should not store any objects in these tablespaces.
• The USERS tablespace is helpful for ad-hoc users.
• The UNDOTBS1 holds the undo data.
• The TEMP is the temporary tablespace which is used for storing intermediate results of sorting, hashing, and large object processing operations.

MySQL

Only the InnoDB engine supports tablespaces, as follows:

• The System Tablespace
• File-Per-Table Tablespaces
• Undo Tablespaces

DB2

When you create a new database, the database manager creates some default tablespaces for the database. These tablespaces are utilized as a storage for user and temporary data. Each database must contain at least three tablespaces as given here:

• Catalog tablespace
• User tablespace
• Temporary tablespace

Going Forward

That concludes the second instalment on tablespaces. This instalment provided some information on what tablespaces are, how they work and the types of default tablespaces you'll find in the various relational database products. The next and final part of the series will focus on how to manage tablespaces in Navicat 16.

The Advantages

Did you know that Navicat 16 supports tablespaces? A table space is a storage structure for tables (as well as indexes, large objects, and long data) that organizes database data into logical storage groupings that relate to where data is stored on the filesystem. It's main function is to link the physical storage layer and the logical storage layer. By assigning tables to a tablespace you can control the physical storage layout by putting some tables on faster or more redundant disks, or to stripe tables across disks. This series is split into two parts: in the first couple of blogs, we'll cover the theoretical side, specifically, what sort of advantages tablespaces offer, as well as how they work and. The second part will focus on more practical matters, i.e., how to manage tablespaces in Navicat 16.

Some Advantages Offered by Tablespaces

Besides the advantages mentioned above, tablespaces offer a few other benefits:

Recoverability

Putting objects into the same tablespace makes backing up and restoring the database easier, since you can backup or restore all the objects within a tablespace with a single command. Moreover, if you have partitioned tables and indexes that are distributed across tablespaces, you have the option of backing up and/or restoring only the data and index partitions that reside in a given tablespace.

Easy to Add More Tables

Although there are limits to the number of tables that can be stored in any one tablespace, should you have a need to store more tables than can be accommodated within a single tablespace, you can easily create additional tablespaces for them using the CREATE TABLESPACE command:

CREATE TABLESPACE tbs1    DATAFILE 'tbs1_data.dbf'    SIZE 1m;

Automatic Storage Management

Usually, you need to define and manage the tablespace containers yourself. However, certain databases, such as DB2, support automatic storage tablespaces, whereby storage is managed automatically. Creating a tablespace with the automatic storage tablespace option delegates the creation and management of containers to the database manager.

Ability to Isolate Data in Buffer Pools for Improved Performance or Memory Utilization

If you have a set of objects (for example, tables and indexes) that are queried frequently, you can assign the tablespace in which they reside to a buffer pool with a single CREATE or ALTER TABLESPACE statement. Temporary tablespaces can also be assigned to their own buffer pool to increase the performance of certain operations such as sorts or joins. For seldom-accessed data, or for applications that require very random access into a very large table, it might make sense to define smaller buffer pools; the data can be kept in the buffer pool for no longer than a single query.

Going Forward

This first instalment of the Navicat 16 and Tablespaces series presented several advantages offered by tablespaces. In the next blog, we'll learn more about how tablespaces work. Finally, we'll move on to working with tablespaces in Navicat 16.

As you well know, multiple server hits can slow down an application. For that reason, developers are keen to find the most efficient ways to update data using as few statements as possible. As it turns out, the SQL UPDATE statement does support the setting of fields from multiple tables using this syntax:

UPDATE table1, table2, ...    SET column1 = value1,        column2 = value2,        ...[WHERE conditions]

The syntax gets formed by the combination of various keywords that helps in the grouping of two or more tables, like the join keyword.

Today's blog will present an overview of the multi-table UPDATE statement along with an example using MySQL 8 and Navicat Premium 16.

Some Caveats

Combining two table updates into one statement is not without limitations and quirks. Here are some points to keep in mind:

• In the multi-table UPDATE query, each record satisfying a condition gets updated. Even if the criteria are matched multiple times, the row is updated only once.
• The syntax of updating multiple tables cannot be used with the ORDER BY and LIMIT keywords.

So, while the multi-table UPDATE statement is quite efficient, it is not ideal for every situation.

A Practical Example

To give the multi-table UPDATE statement a try, we'll create two tables named "library" and "book" and consider the case when one or more books are borrowed from the library. Doing so increases the count of books while decreasing the count of the books. As it turns out, that's the ideal scenario to combine two separate statements into one UPDATE query. This will avoid separate calls to the server, making it a very efficient operation.

Here are the definitions and contents of each table:

The library Table

The book Table

Here is the query that will update both tables:

UPDATE library l, book b    SET l.book_count = l.book_count - 2,        b.book_count = b.book_count + 2WHERE l.id = b.book_idAND b.id = '1AG';

In the above query, the l.id = b.book_id condition acts as an inner join which combines the two tables and operates on the combined table after checking the table constraints. Meanwhile, the b.id = '1AG' condition further reduces the target rows to those which pertain to user '1AG'.

Other join types like outer join and right outer join may be employed as well; the only mitigating factor is that the two tables getting grouped must have a similar/matching attribute.

As with the regular (single table) UPDATE statement, the SET keyword is used along with the UPDATE keyword to set the new values in existing rows. It causes the older values to be overwritten with the new data. We can observe the query results in Navicat below:

As expected, the book counts for user '1AG' and book 103 have been updated in both tables:

Conclusion

Today's blog presented an overview of the multi-table UPDATE statement along with an example using MySQL 8 and Navicat Premium 16. The lesson here is that the multi-table UPDATE statement works best for applying mathematical operations such as incrementing and decrementing on related table columns.

In the Joins versus Subqueries: Which Is Faster? blog article we learned that joins tend to execute faster than subqueries. Having said that, it's not a universal rule, so you may not want to automatically assume that a join will be preferable. As mentioned in that article, if you need to add many joins to a query, the database server has to do more work, which can translate to slower data retrieval times. This article will present a couple of quick tests you can perform to compare a query that employs joins to one that contains subqueries so that you can choose which performs best.

Two Queries, Same Result

Most of the time, a query can be written using joins or subqueries. To illustrate, here is a query that selects countries, along with their associated cities and addresses from the MySQL Sakila Sample Database. The first SELECT statement uses joins while the second one fetches the exact same data using subqueries:

SELECT    co.Country,    COUNT(DISTINCT ci.city_id) AS city_cnt,    COUNT(a.city_id)           AS address_cntFROM country coINNER JOIN city ci    ON co.country_id = ci.country_idINNER JOIN address a    ON ci.city_id = a.city_idGROUP BY    co.country_id;SELECT     Co.Country,    (Select COUNT(1)  FROM City Ci  WHERE Ci.country_id=co.country_id) AS city_cnt,    (Select COUNT(1)  FROM Address A    INNER JOIN city c on a.city_id=c.city_id  WHERE C.country_id=co.country_id) AS address_cntFrom Country Co;

We can easily compare the results in Navicat, because it can run multiple queries simultaneously. Each result set is shown in its own tab below the SQL Editor. In the image below, the contents of the Result 2 tab is shown next to Result 1 for quick comparison:

Query Execution Time

Having verified that both statements are equivalent, we can now compare their execution times.

To do that, we can select an individual statement, and click the Run button, whose label changes to Run Selected whenever text is selected in the editor. An Elapsed Time of 0.020s can be seen at the bottom of the screen:

Doing the same with the second statement yields an Elapsed Time of 0.021s. A minute difference, but one that would grow as the volume of data increases:

Comparing Execution Plans

A query's Execution Plan can reveal a lot of information about how quickly it will execute. In Navicat, we can view the Execution Plan by clicking the Explain button. While it takes some practice to become adept at interpreting the results of Explain, doing so can pay dividends when trying to ascertain a query's efficiency.

The Explain1 tab shows the Execution Plan for the first (join) query. We can see at a glance that it involves 3 SIMPLE selects:

Meanwhile, the Explain2 tab lists one PRIMARY select, followed by three DEPENDENT SUBQUERIES. Even without digging deeper, we can already see that there is an additional step required to execute the second (subquery) statement:

Conclusion

While this blog seems to confirm the conclusion reached by Joins versus Subqueries: Which Is Faster? article, it can be a worthwhile exercise to compare both a join and subquery approach. In any event, there are still times that a subquery is advantageous over joins, such as when you have to calculate an aggregate value on-the-fly and use it in the outer query for comparison.

Back in 2020, we learned about The NULL Value and its Purpose in Relational Database Systems. As stated in that article, the value NULL has become a special marker to mean that no value exists. You could also say that NULL values may indicate that a column could have a value, but you don't know what that value should be yet. In that context, they act as a placeholder until you finally collect the data needed to fill the table field with a real value.

Moreover, when you consider that all major database vendors support NULLs as default values, it only makes sense to use them, doesn't it? Well, not so fast. There are database designers who avoid using NULLs unless absolutely necessary. Do they know something that the rest of us don't? Read on to find out!

Space Considerations

Although NULL values represent "nothing" or "no value", they are treated as a value by the database. As such, they take up space on the hard drive. So, if you think that you are saving hard drive space by employing NULL values, you could be mistaken. In fact, NULL is considered to be a variable-length value, meaning that it could be a few bytes or several bytes, depending on the column type. The database leaves room for extra bytes should the value be larger than what is stored in the field, the result being that your database might take up more hard drive space than if you had used regular values.

Don't Create a Record with Missing Information

Some database administrators argue that if all the columns of a record can't be filled, then a record shouldn't be created. This argument obviously doesn't apply to all use cases, but the idea behind it is that a record should only be created when all fields have actual values without any placeholders. For example, in a banking application, you wouldn't proceed with a transaction if you didn't know the amount of the transaction. Fair enough, but this type of rigorous standard doesn't work so well in other industries such as e-commerce or websites that collect user data.

Complex SQL

Another disadvantage affects your database stored procedures. While most databases provide functions to detect NULL values, special care must still be taken to distinguish NULLs from other values. This means that your SQL procedures might be much longer than necessary, and they can become complex to read as well. A database administrator may reject code changes if the procedures are too convoluted and/or unintelligible.

Case in point, here's a small table in Navicat Premium 16 that contains a combination of values, empty strings, and NULLs:

In Navicat, it's easy to insert an empty string or NULL via the Edit menu.

Now here's a query that counts the number of names based on a variety of criteria:

We were looking for a count of 5 as records 4, 5, 7, 8, and 10 do not have values in them. However, only the combo_count returned 5. This is because while a NULL value does NOT have a length, so NULLs are not picked up by the length() function.

From this example, we can conclude that allowing NULL values can make you work extra hard to get at the kind of data you are looking for. Moreover, allowing NULL values may reduce your confidence regarding the data in your database, as you can never quite be sure whether a value exists or not.

Conclusion

Most database practitioners choose to allow some NULL values in their database tables, as they are the default value in just about any well known database and function well as a placeholder for missing data. On the other hand, we saw here that some DBAs don't feel that NULLs are worth the extra trouble they entail. The moral of this story is that you should consider your own business processes before designing your database(s) and choose a structure that best suits your data.

Although there are few database administrators (DBAs) who do not believe in performing regular database backups, there are many opinions on how best to do so. Whichever approach you espouse, there are many good reasons to keep a copy of the database schema. In the event of data loss, you can restore the database structure from the schema, and then populate it with the latest data backup.

Some database vendors, such as MySQL, offer free utilities (i.e. mysqldump) for backing up the database structure on its own, while others require a specific administration tool to do so. If you're a Navicat user, there's no need for external tools. While data backups may be performed using the Backup Wizard, the schema can be copied using the Data Transfer Tool. In this blog, we'll learn how!

About the Data Transfer Tool

The Navicat Data Transfer Tool is a wizard-driven process that helps you to transfer tables, collections or other objects from one database/schema to another, or to a SQL/script file via a series of screens. The target database/schema can reside on the same server or on a remote server. In Navicat Premium, you can also transfer objects across server types, e.g. from MySQL to SQL Server. Only MongoDB does not support transferring to other server types, due to it being a NoSQL document database, as opposed to a traditional relational database.

You'll find the command to launch the Data Transfer Tool under Tools > Data Transfer in the Main Menu:

Source and Target Screen

The first screen is where you provide the Source Connection and Database/Schema and Target. The Target may be another connection or an SQL File that you can execute to rebuild the database schema later.

We'll specify the File option, and choose a location and name for the SQL/script file:

Options Screen

At the bottom of the Source and Target Screen, you'll see a button for choosing various options, including Table, Record, and Other options.

To backup the database structure only, we simply need to uncheck the Create records option as shown in the image below:

Database Objects Screen

On the Database Objects Screen, we can choose which tables, views, procedures/functions, and events to backup. If we do not select anything here, an empty database will be backed up, without any objects.

Summary Screen

The last screen in the process provides a summary of your choices along the way, so that you can verify them before clicking the Start button. Should you change your mind about anything, you can click the Back button to return to the relevant screen.

You'll also find a couple of common options there for quick selection:

Progress Screen

The Progress Screen displays every step of the backup along with a summary of transferred objects, errors, and the elapsed time:

Conclusion

Keeping a copy of the database schema is always a good idea so that, you can restore the database structure from the schema, and then populate it with the latest data backup, in the event of data loss. Although some database vendors, such as MySQL, offer free utilities (i.e. mysqldump) for backing up the database structure on its own, an even easier option is to use Navicat 's Data Transfer Tool. It can transfer tables, collections or other objects from one database/schema to another, or to a SQL/script file via a series of screens!

Last week's article shed some light on the Outer Joins in SELECT queries. It's a JOIN type that returns both matched and unmatched rows from related tables. Unfortunately, it is not supported by all database (DB) vendors, including MySQL. But that's OK, because Outer Joins can be emulated by combining three other JOIN types, namely LEFT, INNER, and RIGHT joins. In this article, we'll learn more about LEFT and RIGHT joins and how, when combined with an INNER JOIN, they create an OUTER JOIN.

The LEFT Join

The LEFT JOIN returns all rows from the left table and the matching rows from the right table. If no matching rows are found in the right table, a NULL is returned. Here's the syntax:

SELECT    select_listFROM    T1LEFT JOIN T2 ON    join_predicate;

The following VEN diagram illustrates what data is fetched from two tables T1 and T2 using the LEFT JOIN clause:

The RIGHT Join

The RIGHT JOIN returns all rows from the right table and the matching rows from the left table. If no matching rows are found in the left table, a NULL is returned. Here's the syntax for that:

SELECT     select_listFROM     T1RIGHT JOIN T2 ON join_predicate;

The following VEN diagram illustrates what data is fetched from two tables T1 and T2 using the RIGHT JOIN clause:

Combining Joins to Emulate an OUTER JOIN

It is common knowledge throughout the database community that MySQL lacks support for FULL OUTER JOIN. One common workaround to this short-coming is to use a UNION ALL to combine three result sets from a LEFT JOIN, an INNER JOIN, and a RIGHT JOIN of two tables, where a join_column IS NULL condition is added to the LEFT and RIGHT joins.

To demonstrate how to emulate an OUTER JOIN as described above, we'll write a query against the same Project Management database as last week in the Understanding SQL Outer Joins article, but in MySQL this time.

Finding Unmatched Records in the Left Table

This first query will return rows that are found only in the left table. The query below achieves this effect by using LEFT join with a WHERE clause that specifies that the common (joining) column in the right table is null:

Finding Unmatched Records in the Second Table

The second query will return rows that are found only in the right table. To do that, we'll use a RIGHT join with a WHERE clause that specifies that the common (joining) column in the left table is null:

Finding Matched Records in Both Tables

To find records that appear in both tables, we can use a standard (INNER) JOIN like the following:

When combined using UNION ALL, the three separate queries produce the same results as an OUTER JOIN:

Conclusion

In this article, we learned more about LEFT and RIGHT joins and how, when combined with an INNER JOIN, they create an OUTER JOIN. Like last week, there is again a caveat. This technique can be quite inefficient on large tables when used with ORDER BY and/or LIMIT queries, as these utilize a filesort. In such cases, you may want to employ another approach.

The Outer Join is the least understood of all the SQL Join types. Perhaps it's because Outer Joins are required somewhat less often than other join types. In any case, there is nothing inherently strange about Outer Joins. As we'll see in this blog article, a few examples of the Outer Join in action should be enough to clarify any misapprehensions and/or confusion you may have about them.

This blog will first describe the syntax and purpose of the Outer Join statement, which will then be followed by some illustrative examples.

OUTER JOIN Syntax

The OUTER JOIN (or FULL OUTER JOIN if you like) keyword returns all records of two joined tables when there is a match in either the left (table A) or right (table B) table records. The following VEN diagram depicts the potential matches and OUTER JOIN syntax:

Hence, FULL OUTER JOIN returns unmatched rows from both tables, as well as matched rows in both tables. In other words, rows are returned by the query regardless of whether or not the join field (Clave) value is matched across both tables.

Still confused. Don't worry, we'll go over some examples in the next section to clear things up.

OUTER JOINs In Practice

In this tutorial we will use the well-known Northwind sample database.

The following SQL statement selects all customers, and all orders:

SELECT Customers.CustomerName, Orders.OrderIDFROM CustomersFULL OUTER JOIN Orders ON Customers.CustomerID=Orders.CustomerIDORDER BY Customers.CustomerName;

One of the hallmarks of a result set produced by an OUTER JOIN query is that you will see Null values in either joined columns as one may appear in one table, but not the other. We can observe that here in this screen capture of the above query and results in Navicat Premium 16:

Or course, you will never see Nulls in both table columns because a value must appear in at least one table. It should also be noted that the present of a Null in the ContactName columns is problematic because it means that an order was placed that is not associated to an existing customer. This would point to a flaw in the database design, most likely a missing Foreign Key Constraint.

Our second example fetches data from a Project Management database, namely project managers and projects. Here's the SQL:

SELECT     m.name member,     p.title projectFROM     pm.members m    FULL OUTER JOIN pm.projects p         ON p.id = m.project_id;

Again, we can see Null values (at least one Null)

In this case, the results indicate that Jack Daniel has no projects at the moment. Whether or not this represents an issue would depend on that organization's particular operations. It may be perfectly reasonable for a Project Manager to be without a project, or for projects to be unnasigned, at any given time.

Conclusion

Hopefully today's blog helped shed some light on the purpose and usage of Outer Joins in your queries. One final word of warning: Outer Joins can result in very large result sets, so use them sparingly, and include filtering clauses such as WHERE to minimize the number of rows returned.

In the realm of Information Technology, or IT as it's more commonly known, an enum is a special data type that encapsulates a set of predefined constants. As such, the variable may only hold one of the values that have been predefined for it. Common examples include compass directions of NORTH, SOUTH, EAST, and WEST or the days of the week.

One of the complicating factors when storing enums in a database table is that their values may be numeric or alphabetic (i.e. strings). Moreover, you'll want to prevent users from adding any values to the table that are not part of the permissible enum set. We'll be addressing both of these issues in today's blog.

Enum Values Explored

The most basic enums contain a set of zero-based ordinal values each represented by a constant, seen below in Java:

public enum Day {    SUNDAY, MONDAY, TUESDAY, WEDNESDAY,    THURSDAY, FRIDAY, SATURDAY }

More complex enums may also contains other types; strings are the most common, but more complex objects are also supported. Here is an enum for representing different environment URLs (also in Java):

public enum Environment {    PROD("https://prod.domain.com:1088/"),     SIT("https://sit.domain.com:2019/"),     CIT("https://cit.domain.com:8080/"),     DEV("https://dev.domain.com:21323/");     private String url;     Environment(String envUrl) {        this.url = envUrl;    }     public String getUrl() {        return url;    }}

Generally, it is considered bad practice to store enumerations as numerical ordinal values, as it makes debugging and support difficult. It's usually preferable to store the actual enumeration value converted to string. To illustrate, imagine that we had an enum of card suits:

public enum Suit {   Spade,   Heart,   Diamond,   Club }

Now imagine that you are a database practitioner trying to decipher either of the following query results:

Name          Suit------------  ----John Smith    2Ian Boyd      1Name          Suit------------  -------John Smith    DiamondIan Boyd      Heart

I think that you will agree that the latter is much easier to interpret as the first option requires getting at the source code and finding the numerical values that were assigned to each enumeration member.

Although storing strings takes more disk space, enumeration member names tend to be short, and hard drives are cheap, making the trade-off worth while to make your day-to-day job easier.

Another problem with using numerical values is that they are difficult to update; you cannot easily insert or rearrange the members without having to force the old numerical values. For example, adding a value of Unknown to the Suit enumeration would require you to update it to:

...in order to maintain the legacy numerical values already stored in the database.

Validating Enum Values In the Database

Many modern databases, including MySQL and SQL Server, support the ENUM data type. Specified as strings, ENUM values are automatically encoded as numbers when stored for compact storage.

Here are the MySQL statements in Navicat for MySQL to create and populate a table with shirts and their sizes, as well as a SELECT query that fetches medium sized shirts:

If we now try to insert an invalid ENUM value, we get the following error:

Although the message states the the value was truncated, the data was not actually inserted.

Conclusion

In today's blog, we explored how to with enum values in the database, including how to store, validate, insert, and retrieve them.

Interested in trying Navicat for MySQL? You can use it for 14 days for free!

Strings as Primary Keys

In this third and final installment of this series on choosing a Primary Key for relational databases we'll be examining some of the reasons for employing string data as a Primary Key (PK). Recall that, in Part 1, we covered Natural and Surrogate Primary Keys and considered why one might choose one over the other. Part 2 explored String and Numeric data types as Primary Keys in an effort to ascertain whether one is preferable to the other. Now it's time to set the record straight and conclude whether or not string - or alphabetic - data can make a suitable PK.

Ready-made Keys

There is often a sensible natural primary key for your data which has a universal meaning and may not be an integer. If so, then adding an artificial key just for the sake of an integer type adds nothing but redundancy. Performance may be hampered slightly, but it's slightly less important than correctness, integrity, and appropriate modelling, in the estimation of many database developers.

A non-obvious benefit of alphabetic keys is that a short symbolic string can simplify debugging by being immediately human-readable in data dumps (without additional joins). For example, US states have an alphabetic code which is unique and is meaningful as a key outside the schema. Also, countries have alphabetic ISO codes. And there are an endless number of other examples, such as vehicle VIN numbers, invoice IDs, etc.

Using GUIDs as Primary Keys

If you have more than one database, auto-incrementing keys can cause redundant records. One way around this problem is to use GUIDs. Short for "Globally Unique IDentifier", GUID is a 16 byte binary data type that is guaranteed to be unique across tables, databases, and even servers.

How you create a GUID varies across different databases, but, in SQL Server, the NEWID() function is used as shown below:

SELECT NEWID()

Here's a statement for creating a table with the UNIQUEIDENTIFIER data type. To set a default value for the column we will use the default keyword and set the default value as the value returned by the NEWID() function:

USE EngDBGO CREATE TABLE EnglishStudents1(Id UNIQUEIDENTIFIER PRIMARY KEY default NEWID(),StudentName VARCHAR (50) )GO INSERT INTO EnglishStudents1 VALUES (default,'Shane')INSERT INTO EnglishStudents1 VALUES (default,'Jonny')

This will ensure that whenever a new record is inserted in the EngStudents1 table, by default, the NEWID() function generates a unique value for the Id column. When inserting the records, we simply have to specify "default" as value for the first column. This will insert a default unique value to the Id column.

GUIDs vs. UUIDs

While GUIDs (as used by Microsoft) and UUIDs (as defined by RFC4122) look similar and serve similar purposes, there are subtle-but-occasionally-important differences. First, let's establish what UUIDs are.

UUIDs are 128 bits values, with textual representation in hex digits. It bears repeating, because many people think that UUIDs are stored as text. UUIDv4 values being random, you don't have a guaranteed uniqueness. However, the probability of a collision is rather small. Also, keep in mind that in the extremely unlikely case of colliding UUIDs, it will be caught by the DB thanks to the primary key constraint. You'll also be happy to know that UUIDv4 are perfectly well indexed by most relational DBs.

Some Microsoft GUID docs allow GUIDs to contain any hex digit in any position, while RFC4122 requires certain values for the version and variant fields. Also, GUIDs should be all-upper case, whereas UUIDs should be "output as lower case characters and are case insensitive on input". This can lead to incompatibilities between code libraries.

You could say that GUID is Microsoft's implementation of the UUID standard. Treat them as a 16 byte (128 bits) value that is used as a unique value. In Microsoft-speak they are called GUIDs, but call them UUIDs when not using Microsoft-speak.

The Verdict

In this third and final installment of this series on choosing a Primary Key for relational databases, we set out to decisively conclude whether or not string - or alphabetic - data can make a suitable PK. The answer is YES, but with some caveats. You should expect to take a slight performance hit. If that prospect does not deter you, then you've got some options to work with, from short symbolic strings to GUIDS and UUIDs. Finally, try to use fixed length strings rather than varchars, as fixed length strings perform better.

String vs. Numeric Data Types as Primary Keys

Welcome back to this series on choosing a Primary Key for relational databases. In Part 1, we covered Natural and Surrogate Primary Keys and considered why one might choose one over the other. Today's instalment will explore String and Numeric data types as Primary Keys in an effort to ascertain whether one is preferable to the other.

String and Numeric Data Types in Relational Databases

Both string and numeric nomenclatures are actually umbrella terms that encapsulate several different data types. For starters, the string data type is a generic IT term that traditionally refers a sequence of characters, either as a literal constant or as some kind of variable. With regards to databases, single characters, represented by the CHAR type, are also grouped with Strings. Other DB string data types include VARCHAR, BINARY, VARBINARY, BLOB, TEXT, ENUM, and SET. Numeric data types include both exact numeric data types such as INTEGER, SMALLINT, DECIMAL, and NUMERIC, as well as the approximate numeric data types like FLOAT, REAL, and DOUBLE PRECISION.

The Great Debate

Advice on what data type works best for primary keys (PKs) abounds on the Internet. Some sites state outright that numeric keys are almost always superior to character-based ones, while an equal number of sites promote the use of string types. Meanwhile, DB vendors themselves don't suggest one type over the other. What they do offer, are instructions regarding the PRIMARY KEY Constraint. It uniquely identifies each record in a table and posits that:

• Primary keys must contain UNIQUE values, and cannot contain NULL values.
• A table can have only ONE primary key; and in the table, this primary key can consist of single or multiple columns (fields).
• PK values should not be changed over time.

So long as your PK satisfies the above criteria, then you're good to go, as far as DB vendors are concerned. But that doesn't mean that one type may offer some advantages over the other. Let's dive into those now.

Say Aye for Numeric Types

Back when I was first learning about database development, I was instructed that numeric types are best for PKs because they are both faster and memory efficient. This opinion was reinforced by my first employer, the Federal Government, who utilized numeric PKs, even if that meant adding a surrogate key.

There are plenty of reputable reference sites who echo that sentiment. Speaking about MySQL, Mysqltutorial.org states:

Because MySQL works faster with integers, the data type of the primary key column should be the integer e.g., INT, BIGINT. And you should ensure sure that value ranges of the integer type for the primary key are sufficient for storing all possible rows that the table may have.

MySQL is far from unique on its handling of numeric data; another page on Primary Keys in Oracle states that "primary keys typically are numeric because Oracle typically processes numbers faster than any other data types."

They even go so far as to say that PK data should be "meaningless":

Sometimes, you may want use meaningful data, which considers being unique, for the primary keys e.g., social security number (SSN), vehicle identification number (VIN), email, and phone number. However, you dont know when the email or phone number changes or is reused by another person. In such cases, it will create many data problems. In the database world, the artificial keys are known as surrogate keys which are as opposed to natural primary keys.

Coming Up Next Week...

So far, it would seem that Numeric primary keys are best. However, we have not yet heard from the pro-string side. Perhaps they can offer some very good reasons for using strings instead.

Natural vs. Surrogate Keys

One of the first decisions you'll be faced with as a database designer is what kind of Primary Key (PK) to use on your tables. If you ask anyone who works with databases on a daily basis, whether database administrator, developer, or tester, you'll get a myriad of opinions and justifications to go along with them. Compounding the impediments to coming up with an answer is that there is no one size fits all solution. With that in mind, this series will present some reasons both for and against different types of PKs. Somewhere in all those ideas, there will be a few that will steer you towards the best type of PK to use for your organizational needs. In this first instalment, we'll compare the two basic types of PKs: Natural and Surrogate Keys. Later, we'll cover the questions of whether or not to use the database Auto Increment feature as well as which data type(s) - if any - make the best PKs.

Natural Keys

A natural key is one made up of one or more columns that already exist in the table (e.g. they are attributes of the entity within the data model) that uniquely identify a record in the table. Since these columns are attributes of the entity they inherently possess business meaning. The following is an example of a table with a natural key in Navicat Premium 16's Table Designer. We can easily identify the Primary Key by the key icon in the Key column:

Looking at the data, we can see that the productCode has business meaning:

Surrogate Keys

A surrogate key (or synthetic key, pseudokey, entity identifier, factless key, technical key, etc!) is a system generated (GUID, sequence, unique identifier, etc.) value with no business meaning that is used to uniquely identify a record in a table. The key itself could be made up of one or multiple columns (i.e. Composite Key) as well. We can see a surrogate key in a table from the same database, which defines a customerNumber column as its PK:

Although not Auto Incrementing, it's a numeric field that is unrelated to the customer entity:

Making a Decision

So why does one table employ a Natural Key while the other utilizes a Surrogate Key?

It's quite common for products to have some sort of unique inventory number, which makes an ideal PK. Adding an additional numeric key would simply be a waste of disk space and would almost certainly require an additional index on the productCode column for searching. On the other hand, customers don't typically come with unique identifiers. Speaking as someone who has had to uniquely identify persons in a database, it takes a surprisingly long list of columns to do so. Hence, it's usually much easier to assign a numeric Surrogate Key that index every column in the table.

Conclusion to Natural vs. Surrogate Keys

In this first instalment on Choosing a Primary Key, we explored Natural and Surrogate Primary Keys and considered why one might choose one over the other. It's important to decide between using a Natural or Surrogate first because which you choose will help answer some of the follow-up questions as well - especially in the case of a surrogate key.

If you'd like to give Navicat 16 a test drive, you can download a 14 day trial here.

Application developers have long held the belief that housing database operations within stored procedured yielded optimum performance and guarded against SQL Injection attacks. It was also thought that these advantages were worth the extra costs associated with maintenance, testing, and migration of database logic to a different vendor. In recent years, the tide has been turning away from stored procedures - or procs - towards Object-relational Mappers (ORM) such as Hibernate or Entity Framework as developers have begin to question these long-standing assumptions.

The Are Stored Procedures an Outdated Tool? article highlighted a few reasons for eschewing stored procedures in favor of application code and ORMs. This week, we'll explore the two myths introduced above and see if they still stand up to scrutiny today.

Performance Advantages

In the early days of the Internet, it was common practice to minimize network traffic in order to boost performance. Stored procedures helped reduce network traffic by requiring only the proc name and parameters to be transferred over to the server rather than the full SQL statement. Considering the complexity and length of some production queries, these gains could sometimes be substantial. Today, whatever gains you may garner from this approach are easily offset by the fact that you are all too likely to end up calling the same procedure two or three times with the same parameters in the same request. Meanwhile, an ORM would look in its Identity Map and recognize that it already retrieved that result set, so there's no need to do another round trip. Moreover, it should be noted that the claim that stored procedures are cached on the server, whereas ad-hoc SQL is not, is a myth that was busted by Frans Bouma in his blog post, Stored Procedures are bad, m'kay?.

Stored Procedures and SQL Injection

It has often been said that stored procedures offer natural protection against SQL injection because they separate data from instructions. This is true, as long as the developer doesn't use dynamic SQL within the stored procedure where a raw string is passed via the input parameter that replaces the placeholder. Here's a badly written proc that shows exactly how it could open up the database to SQL injection:

create procedure GetStudents(@School nvarchar(50))asbegin    declare @sql nvarchar(100)    set @sql = 'SELECT STUDENT FROM SCHOOL WHERE SCHOOL LIKE ' + @School    exec @sqlend

You can write SQL that eliminates SQL injection vulnerabilities by using parameterized queries. Written in a programmatic language such as python, TypeScript, or Java, a prepared statement like the one below can sanitize user input so that it's safe to use in your queries:

String sql = "SELECT STUDENT FROM SCHOOL WHERE SCHOOL LIKE ? ";PreparedStatement prepStmt = conn.prepareStatement(sql);prepStmt.setString(1, "Waterloo%");ResultSet rs = prepStmt.executeQuery();

The lesson here is that protection against SQL injection is not a benefit of stored procedures themselves, but rather the convention of not concatenating SQL strings together.

Going Forward

This blog explored a couple of long-held assumptions about stored procedures that don't quite hold true today. While not by themselves sufficient reason to hop off the stored procedure bandwagon, they do strongly suggest that it may be time to reevaluate your application architecture.

Data Synchronization

You are now able to choose not to preview the results and deploy directly. Navicat now offers two buttons:

• Compare & Preview: preview the comparison results.
• Compare & Deploy: skip the preview and deploy immediately.

Dump SQL File

Weve enhanced our Dump SQL File feature. The file created from Dump SQL File can now be opened in three ways:

• Open: open the file using Query Editor.
• Open with External Editor: open the file with different editors.
• Open Containing Folder: open the folder where the file located.

Stored procedures have been falling out of favour with some organizations for several years now. The preferred approach of these businesses for accessing their database(s) is to employ an Object-relational Mapper (ORM) such as NHibernate or Entity Framework. Over the next couple of blog articles, we'll explore their reasons for doing so, and whether this paradigm shift points to the eventual obsolescence of Stored Procedures.

Stored Procedure Basics

As expressed in the Understanding Stored Procedures and Functions in Relational Databases:

A stored procedure - or "proc" for short - is a set of Structured Query Language (SQL) statements with an assigned name, which are stored in a relational database management system as a group, so it can be reused and shared by multiple programs. Stored procedures can access or modify data in a database, but it is not tied to a specific database or object. This loose coupling is advantageous because it's easy to reappropriate a proc for a different but similar purpose.

Sounds like a useful tool so far, but, as we'll see in the next section, not everyone is convinced.

Drawbacks of Stored Procedures

Despite their long-established advantages, opponents of Stored Procedures point out their many disadvantages, such as:

• Bug Prone: Since stored procedures encapsulate application logic, it should be moved into the application code, where it can be better managed and tested. Due to the inherent challenges in testing stored procs, they can be the cause of some really nasty bugs.
• Implementation Differences: Stored procedure implementations vary from vendor to vendor. While many DB developers consider Oracle's stored procedures to be of the highest quality, other products' procedures, such as those of MySQL, are less well conceived.
• Changing Requirements: One of the original use cases for stored procedures was to reduce network traffic. However, with today's lightning fast network speeds, this isn't nearly as big an issue as it once was. As such, dropping application logic into stored procedures can be a case of premature optimization.
• Difficult to Maintain: Stored procedures tend to require much more work to develop and maintain than application code. For starters, you need to have individual stored procedures to execute create, retrieve, update and delete operations for each table, plus a separate stored procedure for each different query that you wish to make. Then, you need to implement classes and/or methods in your code to call each stored procedure. Compare that with an O/R mapper, where all that's needed are class definitions, database table, and mapping file. In fact, modern ORMs use a convention-based approach that eliminates the need for a separate mapping definition.
• Code Duplicaton: Stored procedures require you to violate DRY (Don't Repeat Yourself) principle, since you have to reference database table columns half a dozen times or more. Moreover, it isn't possible to pass an object as a parameter to most stored procedures - only simple types like string, integer, date/time, etc. - making it virtually impossible to avoid huge parameter lists (a dozen or more is common!).

Even the most staunch opponents of stored procedures still use them in some circumstances. For example, stored procs are great for database housekeeping or reporting. Otherwise, developers should have very good reasons to integrate them into their applications.

Going Forward

Having heard a few reasons for eschewing stored procedures in favor of application code and Object-relational Mappers (ORMs) such as NHibernate or Entity Framework, you may be convinced that this is the way to go. Well, don't make the switch just yet; in the next installment, we'll consider a few more motivations for both abandoning and staying with stored procs. Then, if you still want to make the change, at least you'll be armed with a more complete understanding of all the issues involved.

Charts

Trend Line supported

Starting with Navicat 16.1, trend line can be added in Vertical/Horizontal Bar Chart, Line Chart, Area Chart, Bar and Line Chart, and Scatter Chart. To make your chart even more understandable and easily interpreted, you can change the default appearance of a trendline.For this, just simply click on Trend Line tab in Properties pane.

Better properties pane

The Properties pane allows you to customize your charts. Before, all properties were displayed under one panel only. Now, properties are categorized into several tabs - General, Data, Axis and Trend Line. Each tab regroups the properties and settings, ensuring that you can easily look for the right properties settings when you need them.

Support for additional month format

Weve added the ability to abbreviate the name of the month to one-letter. For example, January appears as J.

Changing chart color

Previously, it was time-consuming to look for the color settings in Properties pane. Its now possible to change the chart color on right-click context menu.

Other improvements

• Weve enhanced our filter function which now dividedinto two filters. The Data Filter is for filtering the source data in the current chart; The Display Filter is applying on the display data.
• Added smoothed line in Line Chart.
• Added "Restore Defaults (Style Only)" on each Properties tab for restoring the chart style only.
• Added hint when resizing/moving objects in dashboard.

Data Viewer

Better view for your data

We think you will particularly appreciate the improvement for highlighting the selected row and active cell. You can now easier to see your current place in Grid View.

Weve improved the data alignment in Grid View. Previously,this made it difficult to view data between number column and text column, but now Navicat handles this situation well.

Show and hide Columns

Weve made it easier to show/hide columns adding a Columns button on the toolbar for quick access.

Table Designer

Heres what we improved for Table Designer:

• Added column CHECK support for MariaDB 10.2.1 or above.
• Added support for PostgreSQL partitioned hash.

Query

Completion in MongoDB

Logical operator is now available in code completion for MongoDB.

Copy code snippets to cloud

When writing SQL, you might sometimes insert reusable code into editor. If you want to share your own snippet in cloud, use a new action, Copy Snippet To, under the Code Snippet pane.

General

Connection tree

• Pressing Ctrl+F in connection tree will now open the Search box in tree. It activated another Search box before.
• Adding shortcut in submenu to have quick access to create a new connection profile.

Information Pane

The behavior of the Information Pane is now more straightforward:

• Highlighting and copying the object title directly.
• Adding the Copy button - make it easierto copy the value to clipboard.
• Showing the related information that you require. Before, SSH/HTTP still displayed even they werent being set.
• Finding in DDL tab - if you are looking for a particular information in DDL, you can use text search for help!

Open currenttabinnew window

You can now right-click on a tab and open it to new window.

Connection

Support OceanBase Community Edition

We announce that OceanBase has been added in our Navicat family. Together with our efficient technical support, Navicat is exactly what you need.

Upgrade for MongoDB Driver

The following points have been improved for MongoDB connection:

• MongoDB driver upgraded from 1.16.2 to 1.21.1.
• Support MongoDB Atlas Serverless.
• You can now choose either driver 1.21.1 (Default) or 1.16.2 (Legacy).

Show Hidden Passwords Behind Dots

We hide the password field with dots for security purposes. You can now click on the eye symbol to reveal the masked password.

Part of knowing your business is tracking sales metrics such as units sold and identifying your best customers. To that end, you'll probably want to begin with fetching data about customers who've made the most purchases throughout the month, quarter, year, or other time period. This data will allow you to analyze their buying patterns and identify trends. This blog will present a few sample queries to do that by combining the mighty Count() function with the GROUP BY and HAVING clauses.

A Basic Query

We'll be executing our queries against the Sakila Sample Database. It's a nicely normalized schema modeling a DVD rental store, featuring things like films, actors, film-actor relationships, and a central inventory table that connects films, stores, and rentals. Hence, its customers are not buying movies, but rather, renting them. Nevertheless, the criteria for selecting the data remains the same, which is to count the rows of the main rental table and group results by customer_id. Here is a basic query in Navicat Premium 16 that limits results to those customers who rented more than 20 movies in total:

That orders results by customer_id. Later on, we'll sort results by num_of_films_rented.

Fetching Additional Customer Details

While the above query is sufficient to identify those customers who rented many movies, it does not provide any customer details other than their IDs. To include more customer data, we need to join the customer table. It should be LEFT JOINed so that only customers who have rented movies are joined to the main query. Here are the results with customer names added:

Filtering Results

So far, we've been casting a very wide net, including results for all films and time periods. We could get more specific by targeting films by category as well as time period. To do that, we'll need to add a few more tables. If you're ever unsure on how to JOIN tables to a query, in Navicat, you can select the tables in the Object pane and run the Reverse Tables to Model... command:

That will add them to a schema diagram in the Modeling Tool so that you can view their relationships:

In the revised query, we'll limit results to comedies that were rented throughout 2005:

Notice that the minimum film count was lowered to 5 or more because there are less rentals for a single category.

Sorting By Count

Perhaps you'd rather view records by rental counts. All that's required to make that happen is to include an ORDER BY clause. Here is the final query, sorted by num_of_films_rented in DESCending order, so that the customer who rented the most comedies in 2005 appears at the top of the results:

Conclusion

In today's blog we learned how to combine the Count() function with the GROUP BY and HAVING clauses to gain valuable insight into our customers' spending habits. As you can imagine, the same query structure can be utilized to discover all sorts of trends and patterns related to product sales and/or rentals. Insights gleaned can be of tremendous benefit in guiding organizational decisions.

Having to select only odd or even rows from a table sounds like something that you'd never have to do, that is until you do. A quick Google search confirms that it's something that is done often enough, but, with few database practitioners knowing how, they invariably turn to online database communities in search of answers. As a reader of this blog, you can save yourself the trouble of scouring database forums for a solution, as we'll set the record straight right here today.

Picking a Suitable Target Column

Before we can speak of "even or odd rows" we have to order the rows by the column whose data we're interested in splitting. Ideally, its data should be numeric, unique, and sorted in ascending order. Hence, auto-increment columns like those of a primary key make perfect candidates. Otherwise, you may need to write a subquery with an ORDER BY clause and then select from it.

As an example, let's open the orders table of the classicmodels sample database in Navicat Premium 16's Table Designer. We can see that its PK (the orderNumber column) is not auto-incrementing, as evidenced by the unchecked "Auto Increment" checkbox:

However, opening the table in Grid View shows that orderNumber values are clearly sorted in ascending order:

Hence, we can write a query directly against the table.

Solutions By Database

The simplest way to find the records with odd or even values is to check the remainder when we divide the column value by 2. A remainder of 0 indicates an even number, while an odd number points to an odd number. However, like so many database tasks, how you go about determining the remainder depends on what type of database you're working with.

In PostgreSQL, MySQL, and Oracle, we can use the MOD() function to check the remainder:

Here's the general query syntax to find rows where a specified column has even values:

SELECT * FROM table_name WHERE mod(column_name,2) = 0;

This syntax will find rows where our target column has odd values:

SELECT * FROM table_name WHERE mod(column_name,2) <> 0;

SQL Server does not have a MOD function. Instead, it provides the % modulus operator.

Here's the general query syntax to find rows where a specified column has even values:

SELECT *FROM table_name where column_name % 2 = 0;

This syntax will find rows where our target column has odd values:

SELECT *FROM table_name where column_name % 2 <> 0;

Some Examples

Let's give each of the above statements a try against the orders table of the classicmodels sample database, first in SQL, then in SQL Server.

First, we'll retrieve even rows:

Next, we'll fetch odd rows only:

As mentioned previously, SQL Server does not have a MOD function, so well employ the % modulus operator instead.

Even rows:

Odd rows:

Conclusion

This blog presented an easy way to retrieve odd or even numbered rows from various databases by checking the remainder after dividing the target column value by 2 - a solution that is both simple and effective.

Supported Platforms and Databases, Plus SQL Editing

In this three part series, we've been comparing HeidiSQL, a free database client, with Navicat Premium. So far we've done a quick visual comparison and looked at both tools' pros and cons. In this final instalment, we'll be examining specific features, such as supported platforms and databases, SQL Editing, and more!

Supported Platforms

HeidiSQL was built for the Windows platform, and still only works on Windows. On the download page, there is a 32/64 bit combined (SHA1 checksum) Installer, Portable (zipped) versions for 32 bit and 64 bit, as well as the full source code. You can run HeidiSQL on Wine. Short for "Wine Is Not an Emulator", Wine is a compatibility layer for running Windows applications on several POSIX-compliant operating systems, such as Linux, macOS,and BSD. On Wine, HeidiSQL runs fine on Windows 8 and 10, but has minor issues on both Windows 7 and 11. Moreover, running HeidiSQL on any Wine release newer than 4.0 is currently quite unstable.

Navicat Premium is available for Windows (32 and 64 bit), macOS (64 bit), and Linux (64 bit). As such, each version of Navicat is optimized for that O/S. As a commercial product, Navicat comes with customer support. Users can also submit a ticket in the rare instance that they encounter a bug, to receive assistance in resolving it and have it fixed in the next patch or minor release.

Supported Databases

Initially, HeidiSQL offered support for MySQL and MariaDB, then added MS SQL Server. Now it includes PostgreSQL support as well.

Meanwhile, Navicat Premium is a Universal Database Tool, which means that it supports all popular databases, including MySQL, MariaDB, MongoDB, SQL Server, Oracle, PostgreSQL, and SQLite. It is also compatible with cloud databases, such as Amazon RDS, Amazon Aurora, Amazon Redshift, Microsoft Azure, Oracle Cloud, Google Cloud and MongoDB Atlas.

SQL Editing

Both Navicat and HeidiSQL's Query Editors are similar in terms of functionality. Both offer auto-completion and customizable Code Snippets that strip the repetition from coding. Here's a side-by-side comparison with HeidiSQL on the left and Navicat Premium on the right:

Each tool includes common SQL statements, functions, code snippets in the right-hand pane. Here they are with HeidiSQL again on the left and Navicat Premium on the right:

One Navicat tool that is absent in HeidiSQL is the Visual Query Builder. It allows anyone to create and edit queries with only a cursory knowledge of SQL:

A Visual Query Builder is a must-have for many users, as is evidenced by a forum thread about that very subject on the HeidiSQL site.

Conclusion

In part 3 of this series on HeidiSQL vs. Navicat Premium, we examined specific features of both products, such as supported platforms and databases, as well as SQL Editing. While both share many similarities, there is no question that Navicat offers a more comprehensive array of tools and features, from the Visual Query Builder, to Navicat Cloud, to a dedicated support team. As we saw when we previously compared DBeaver to Navicat, Navicat Premium perfectly exemplifies the age old adage that you get what you pay for.

General Comparison

While HeidiSQL and Navicat Premium share many similarities, they are in fact different product types. Navicat Premium is what's known as a Universal Database Tool, which means that it supports ALL popular databases, including MySQL, MariaDB, MongoDB, SQL Server, Oracle, PostgreSQL, and SQLite. Moreover, Navicat is compatible with cloud databases as well, such as Amazon RDS, Amazon Aurora, Amazon Redshift, Microsoft Azure, Oracle Cloud, Google Cloud and MongoDB Atlas. HeidiSQL began as a MySQL/MariaDB client and evolved to suport a few additional database types. That being said, the two products are homogenous enough to warrant a side by side comparison. In this installment we'll be taking a high level inventory of pros and cons, while the next part will focus on specific features.

Visual Interface

A quick look at an application's visual interface, or UI/UX, as it's more commonly known, can instantly give us some idea about how easy or difficult the application might be to work with. With that in mind, here's a screen capture of HeidiSQL's Data view:

There is no question that the HeidiSQL UI is chock full of information. Perhaps a little too much, as some elements run out of room at smaller viewport sizes:

Aside from minor glitches, the overall design is excellent; there is a main toolbar for accessing common functionality, and the bottom pane shows all database commands in real time.

Navicat 16 saw a lot of changes to the GUI. In fact, it was completely revamped with the goal of improving the usability and accessibility, allowing the user to accomplish complex tasks faster than ever before:

Navicat shows the latest database command at the bottom of the screen, and also includes additional table, column, and DDL information in the right pane.

Some Quick Pros and Cons

Now, let's run through some pros and cons of each product. First, HeidiSQL:

Pros:

• It's lightweight.
• Connects to multiple servers in one window.
• Free to use Licensed under GNU GPL. The source code is also available.
• Available in a portable version.
• Full database user roles and privileges management.
• Write queries with customizable syntax-highlighting and code-completion.
• Data synchronization. HeidiSQL can compare and synchronize your data and structure between local and remote databases.
• SSH tunneling support

Cons:

• Low stability. HeidiSQL is known to have a lot of bugs that result in frequent crashes.
• It's only available for Windows and it doesn't look like a cross-platform is coming anytime soon.
• No built-in debugger included
• Low DPI display. (DPI stands for dots per inch and it determines clarity and crispness to a display.) The author attempted to add high DPI but wound up dropping it.

And now, Navicat Premium:

Pros:

• It's cross-platform and supports multiple drivers.
• Data and structure synchronization.
• Visual query builder and report builder.
• Excellent import/export capabilities.
• SSH tunneling and SSL (Secure Sockets Layer) support
• Supports many languages, including Polish, Russian, Japanese, Portuguese, Korean, Simplified Chinese, Traditional Chinese, Spanish, French, and English.
• Compatible with other Navicat products, including Navicat Monitor, Navicat Data Modeler, Navicat Report Viewer, and Navicat Data Modeler Essentials.

Cons:

• It's a commercial product. Perhaps a con if you're on a very limited budget and need to work with multiple database types, such as PostgreSQL, SQL Server, or SQLite. In that case, you would have to purchase the Navicat premium package.
• It's somewhat resource-intensive as it requires fairly high memory while running.

Coming Up...

In part 3, we'll be examining specific features, such as supported platforms and databases, as well as SQL Editing.

Meet the Contestants

There is no shortage of either free or commercial relational database clients. Some provide basic functionality, while others offer advanced tools that help professionals achieve their many day-to-day activities in an efficient manner. While there is some correlation between cost and the number of features offered, each product needs to be evaluated on its own merits when deciding which product(s) to use yourself.

Back in June of last year, we compared DBeaver, a popular free tool, to Navicat Premium 15. Now, it's high time that we did it again. In this installment of Database Tools Showdown, we'll be taking a look at another freebie called HeidiSQL, and see how it stacks up against Navicat Premium 16.

Some Product Background

First, the challenger:

Invented by Ansgar Becker in 2002, his aim was to create software that was easy to learn. Becker chose the name "HeidiSQL" when it was suggested to him by a friend as a tribute to Heidi Klum. The name also reflected Becker's own nostalgia for Heidi, Girl of the Alps. "Heidi", as it's affectionately known, lets you view and edit both the data and structures of MariaDB, MySQL, Microsoft SQL, PostgreSQL and SQLite.

HeidiSQL began as a MySQL front-end in 1999 under the project name "MySQL-Front". In 2004, during a period of inactivity, Becker sold the MySQL-Front branding to his business partner, Nils Hoyer. In April 2006, Becker open-sourced the application on SourceForge, renaming the project "HeidiSQL". Later, he added support for other database servers as follows:

• Microsoft SQL Server support was added in March 2011 for the 7.0 release.
• PostgreSQL was introduced in March 2014 for the 9.0 release.
• SQLite support was introduced in March 2020 for the 11.0 release.

Today, HeidiSQL is routinely ranked among the most popular tools for MariaDB and MySQL worldwide. Since the 8.0 release, HeidiSQL offers its GUI in about 22 languages other than English.

About the reigning champion:

Navicat Premium is Navicat's flagship product. It's a commercial database development and design tool that allows users to simultaneously connect to multiple local and/or cloud databases from a single application. Navicat Premium was designed to meet the needs of a variety of audiences, from database administrators and programmers to various businesses/companies that serve clients and share information with partners.

The initial version of Navicat was developed by Mr. Ken Lin in 2001. The main goal of the initial version was to simplify the management of MySQL instances. In 2008, Navicat for MySQL was the winner of the Hong Kong ICT 2008 Award of the Year, Best Business Grand Award and Best Business (Product) Gold Award. Navicat Premium was launched in 2009. It combined all previous Navicat versions into a single product and could connect to all popular database types simultaneously, giving users the ability to perform data migration between different (heterogeneous) database types.

Going Forward

Now that we've properly introduced our contestants, the next installment(s) will delve into each tool's feature set, and compare them for usability, performance, user ratings, and more!

A CSV is a Comma-Separated Values file, which allows data to be saved in a tabular format. It's long been the preferred format for transferring data between databases. More recently, Internet-driven formats such as XML and JSON have also gained much traction. CSV files are well suited to databases because they represent table data exceptionally well and can be used with just about any spreadsheet program, such as Microsoft Excel or Google Spreadsheets. In today's blog, we'll be taking a look at a few ways to export table data to CSV in MySQL.

Using the Command Line

Most relational databases, MySQL included, provide commands to export and import to and from CSV.

Make sure that you start your MySQL server instance with the secure-file-priv option. It sets the directory where MySQL imports and exports data using statements such as LOAD DATA and SELECT INTO FILE. You can see the current setting using the command:

SHOW VARIABLES LIKE "secure_file_priv"  

All that's left to do now is select the data and specify the location of the output file. Here's a statement that outputs an entire table:

TABLE tableName INTO OUTFILE 'path/outputFile.csv'FIELDS TERMINATED BY ','OPTIONALLY ENCLOSED BY '"'ESCAPED BY ''LINES TERMINATED BY '\n';

You can also filter the data as you would in any SELECT query. Here's an example that filters both columns and values:

SELECT columnName, ...FROM tableNameWHERE columnName = 'value'LIMIT 1000INTO OUTFILE 'path/outputFile.csv'FIELDS TERMINATED BY ','OPTIONALLY ENCLOSED BY '"'ESCAPED BY ''LINES TERMINATED BY '\n';

Want to include column headers? That's easily done using the UNION statement:

(SELECT 'columnHeading', ...)UNION(SELECT column, ...FROM tableNameINTO OUTFILE 'path-to-file/outputFile.csv'FIELDS ENCLOSED BY '"' TERMINATED BY ','ESCAPED BY '"'LINES TERMINATED BY '\n')

Using mysqldump

mysqldump is a command line utility provided by MySQL for exporting tables, databases, and entire servers. Moreover, it can also be utilized for backup and recovery. Issue the following command in a command prompt/terminal to export a table:

mysqldump -u [username] -p -t -T/path/to/directory [database] [tableName] --fields-terminated-by=,

Using Navicat's Export Wizard

Navicat 16 for MySQL comes with a very powerful export (and import) wizard, which can export data in multiple formats, including .xlsx, .json, and .sql. To start the export wizard, select the corresponding table, right-click > Export Wizard, and select the format:

You can choose to export one table, the entire database, or anything in between:

You can also select exactly which fields you want, if you're not interested in all the columns:

Navicat supports a wealth of options, such as including headers, delimiters, error handlers, and more:

Conclusion

CSV is not the perfect format, and does have limitations. For example, you cannot save data types or formulas in this format. That being said, CSV is still a very important data transfer format; one that every DBA should be familiar with.

In some cases, running a well crafted UPDATE statement in production can save the day. Other times, a botched UPDATE can cause more harm than the initial issue. You can always execute your Data Manipulation Language (DML) statements on a development or test database, but due to differences in the data, this approach makes determining the statement's effects on the production data a craps shoot at best.

So what are some options to accurately predict what the result of an INSERT, UPDATE, or DELETE statement will be on production data before running it? Well, that depends on the database vendor and product, at least in part. There are also some solutions that enjoy widespread support. We'll be taking a look at both options in this blog.

Syntax Check

The process of testing your statements can be split into two stages. The first is to verify that the statement is syntactically valid, I.E. it will execute. The next step is to ascertain whether or not it produces the result that you intended.

One way to validate your syntax is to ask your database (DB) for the query plan. This tells you two things:

• Whether there are any syntax errors in the query; if so the query plan command itself will fail.
• How the DB is planning to execute the query, e.g. what indexes it will use.

In most relational DBs the query plan command is "explain" or "describe", as in:

explain update ...;

In Navicat's database administration and development tools, you can run the EXPLAIN command with the click of a button. If the statement fails, you'll get an error message similar to the following:

Otherwise, the query plan will be displayed in a tabular format:

Statement Testing

You can parse a statement to see if it's syntactically valid, but that doesn't mean it will produce the correct results. To see what your query will actually do, you've got a few options.

Turn Off Autocommit

Most relational DBs provide a way to disable autocommit mode so that you must issue the COMMIT statement to store your changes to disk or ROLLBACK to ignore the changes.

In MySQL the command to disable autocommit mode is:

SET autocommit=0OrSET autocommit = OFF

In SQL Server, the command is:

SET IMPLICIT_TRANSACTIONS OFF

With autocommit turned off, you are now ready to give your statement(s) a try, by running it within a transaction:

-- 1. start a new transactionSTART TRANSACTION;-- 2. insert a new order for customer 145INSERT INTO orders(orderNumber,                   orderDate,                   requiredDate,                   shippedDate,                   status,                   customerNumber)VALUES(@orderNumber,       '2005-05-31',       '2005-06-10',       '2005-06-11',       'In Process',        145);        -- 3. then, after evaluating the results,--    rollback the changesROLLBACK;

That will leave your DB in exactly the same state as it was before you ran your statement.

Convert Your Statement Into a SELECT

A decidedly low tech approach to testing DML statements is to convert them to SELECTs. As long as you don't expect them to retrieve the entire database, running them as SELECTs is a good way to see exactly which records will be affected. All you need to do is replace the action word with SELECT:

INSERT INTO orders...BECOMESSELECT * FROM ORDERS...

Conclusion

There are few things scarier than executing DML statements in a production environment. Thankfully, there are ways to minimize the risk so that you don't have to cross your fingers or recite the Hail Mary.

If you'd like to give Navicat 16 a test drive, you can download a 14 day trial here.

Just when you thought you knew every type of join, here comes another! Perhaps you've heard of nested joins, or even nested-loop query plans, and wondered what they were. Well, wonder no more. Today's blog will settle the mystery once and for all!

A Case of Terminology

In the world of relational databases, there can be many different names for the same thing. Joins are no exception to this rule. In fact, when it comes to Nested Joins, database practitioners vary as per their opinions on them. Some say that there is no such thing; others are more pragmatic and acknowledge that they are simply an alternative term for multi-table joins.

In all likelihood, the term came about when referring to nested-loop query plans. These are often used by the query engine to answer joins. In its crudest form, a nested loop goes something like this:

for all the rows in outer table   for all the rows in the inner table     if outer_row and inner row satisfy the join condition       emit the rows   next inner next outer

This is the simplest, but also the slowest type of nested-loop. Meanwhile, multi-table nested-loop joins perform even worse because they scale out horrendously as the product of the number of rows in all the tables involved grows.

A more efficient form of nested loop is nested-loop-over-index:

for all the rows that pass the filter from the outer table   use join qualifier from outer table row on index on inner table     if row found using index lookup       emit the rows next outer 

Nested Join Syntax

Now that we've established that the term "Nested Joins" simply refers to joins between more than two tables, let's take a quick look at their syntax.

Typically, when we need to join multiple tables and/or views, we would list them one by one, using this generic format:

FROM  Table1 [ join type ] JOIN Table2                 ON condition2 [ join type ] JOIN Table3                 ON condition3 

But this is not the only way. The official ANSI syntax standard for SQL proposes another valid way to write the above join:

FROM  Table1 [ join type ] JOIN Table2 [ join type ] JOIN Table3                 ON condition3                 ON condition2 

To make the above join style more human readable, we can add parentheses and indentation to make the meaning clearer:

FROM  Table1 [ join type ] JOIN ( Table2                     [ join type ] JOIN Table3                                     ON condition3 )                 ON condition2 

Now it's easier to see that the join between Table2 and Table3 is specified first and has to be done first, before joining to Table1. This query style also positions the join between Table2 and Table3 in such a way that they appear to be nested. In fact, we could consider the join between Table2 and Table3 to be nested.

Here is the same query in Navicat Premium written in both styles:

Conclusion

Today's blog shed some light on the terms "Nested Joins" and "Nested-loop Query Plan". So, next time you hear them, realize that they are merely referring to the joining of multiple tables or views, regardless of which syntax is employed in doing so.

Having completed our series on Top SQL Query Mistakes last week, it's time to take a page from the Monty Python playbook and move on to something completely different. And that something is why database developers and administrators should consider using third-party database administration tools (DBMT) to fill the gaps left by the major database manufacturers. Regardless of price, all 3rd party DBMT provide functionality that fulfills the needs of the general DBA community by either complimenting or replacing database manufacturers' tool sets. Today's blog will highlight just a few of the benefits provided by 3rd party DBMT.

Heterogeneous DBMS Support

It's a fairly rare occurrence to find an IT organization that supports a single DBMS platform these days. Most businesses utilize several different database types - both locally hosted and in the Cloud. For example, my own employer has some local PostgreSQL databases as well as some of Amazon's online DB services. The local DB instances are well suited to development, testing, and certain production uses. Meanwhile, Amazon database services pair well with other Amazon services, such as AI, batch processing, etc...

A growing class of third-party tool providers is capitalizing on this DBMS proliferation by providing tools that are purposely designed to work with a variety of database types. Their value stems from their ability to manage multiple DBMS from a single application interface. Some products, such as Navicat Premium, can even connect to multiple heterogeneous DBMS simultaneously, allowing admins to transfer data between databases as easily as copying files on a PC desktop.

Increased Data Security

An organization's data is invariably its greatest asset. As such, most businesses place a high premium on security and choose 3rd party DBMT that provide more secure connection options than those offered by individual DB vendors' own administration and development tools.

One popular feature is SSH tunneling. It's a method of transporting data that employs an encrypted SSH connection. SSH tunnels allow connections made to a local port to be shuttled to a remote machine via a secure channel.

3rd party DBMT also add value by supporting multiple authentication methods such as PAM authentication for MySQL and MariaDB, Kerberos and X.509 authentication for MongoDB, and GSSAPI authentication for PostgreSQL. High end products like Navicat provide more authentication mechanisms and high-performance environments so you never have to worry about connecting over an insecure network.

Collaboration Support

One feature that is almost universally missing from DB vendor administration and development tools is the ability to share queries and such with team mates. Navicat's main collaboration tool is Navicat Cloud. It uses Amazon Simple Storage Service (Amazon S3) to store (256-bit AES) encrypted connection settings, queries, models, snippets, virtual group information, and even chart workspaces. These may be shared as well as synchronized across all your devices including Windows, macOS, Linux, and iOS. Files stored in Navicat Cloud automatically show up in Navicat so that you can get real-time access at anytime and from anywhere.

Conclusion

This blog presented three ways that 3rd party Database Management Tools such as Navicat Premium provide value to organizations by either complimenting or replacing database manufacturers' tool sets. In doing so, they can greatly simplify everyday tasks and promote increased productivity.

Predicate Evaluation Order

Just before Part 3 of this series, we took a brief pause to talk about Predicates in SQL, as they factored into mistakes related to Outer Joins. In this final installment of this series on Top SQL Query Mistakes, predicates will once again enter the picture, as we examine how predicate evaluation order can cause seemingly well constructed queries to fail with errors.

A Quick Review of Predicate Processing Order

In terms of logical query processing order, queries are executed in the following sequence:

• FROM
• WHERE
• GROUP BY
• HAVING
• SELECT

Hence, logically speaking, the FROM clause is processed first to define the source data set. Next, the WHERE predicates are applied in order to whittle down the result set, followed by GROUP BY, and so on.

In practice, predicate evaluation and processing order is far less rigid, as the query optimizer may move expressions in the query in order to produce the most efficient plan for retrieving the data. As a result, a filter in the WHERE clause may not be applied before the next clauses are processed. In fact, a predicate may be applied much later in the physical execution plan than you might expect.

Another common source of confusion and frustration to database developers is that, unlike with most programming languages, predicates are not always executed from left to right. This means that, if you have a WHERE clause containing the filters "WHERE a=1 AND b=2", there is no guarantee that "a=1" will be evaluated first. In fact, there is no easy way to tell which order filters will be executed simply by looking at the query.

A Practical Example

To better understand predicate evaluation order, we'll write a SELECT query against the following accounts table, seen in Navicat 16's Table Designer:

Here is some sample data that we'll be querying against:

In the account_number column, business accounts are assigned a numeric identifier, while personal accounts are given an identifier made up of characters. This is not great table design, as the account_number column should be represented by two different fields, where each account type is given the correct data type and not share the same table. Nonetheless, altering the design is not always possible, so we must deal with the table as is.

So, with this in mind, let's devise a query to retrieve all business type accounts with an account_number that is greater than 50. The resulting query might look like this one:

In some databases, the query produces an error:

Conversion failed when converting the varchar value 'ACFB' to data type int

The query will fail any time that the query optimizer decides to prioritize the "CAST(account_number AS UNSIGNED INTEGER) > 50" predicate over the "account_type LIKE 'Business%'". The safest be for avoiding errors like the one above is to either:

• Design the table correctly and avoid storing mixed data in a single column.

  OR

• Use a CASE expression to guarantee that only valid numeric values will be converted to INTEGER data type, like this:

  

Conclusion

In this series on Top SQL Query Mistakes, we explored how seemingly intuitive ways of constructing SQL queries can result in anti-patterns that lead to erroneous results and/or performance degradation. Be especially wary of predicate placement and evaluation order as these contribute to many unexpected issues.

Breaking Subqueries

In this series on Top SQL Query Mistakes, we've seen several examples of SQL queries that look perfectly solid on first inspection, but can lead to erroneous results and/or performance degradation. Last week, learned how the placement of predicates can adversely affect query execution - particularly in outer joins. Today's installment will focus on subqueries, and how they can break an SQL statement when changes are made to any of its underlying tables.

Single vs. Multiple Value Subqueries

Even before we compare single and multiple value subqueries, we should briefly cover what a subquery is. A subquery is a complete SQL query that is nested inside a larger query. A subquery may be placed in the SELECT, FROM, and WHERE clauses.

Now that we know what a subquery is and where it can go in a query, it should be noted that, like any SELECT query, a subquery may return one or more rows. This distinction is quite important, because it affects how you would write your query statement. For example, here's a query against the Sakila Sample Database in Navicat Premium 16 that fetches all of the actors who appeared in the film "ALONE TRIP":

Since there should only be one film named "ALONE TRIP", we can use the equals (=) operator to match the film_ids against.

Contrast the above query to the following one:

In this case, the subquery selects all of the actors who appeared in the movie. Naturally, this subquery would return multiple rows. In that case, we should employ the IN() function to match actor_ids against.

How Single Row Subqueries Break

As mentioned earlier, a subquery can be placed in the SELECT clause to fetch a column that is in some way correlated to the main query table. For example, consider these two related products and factories tables, shown in the Navicat Data Modeler:

The products and factories tables are linked using the common sku field.

Now, let's write a query to extract the factory_id for each product. One way to do that would be to write the query using correlated subquery to retrieve the product factory_id:

Note that the point here is to illustrate a technique; there are more efficient ways to retrieve the same information. In any event, we do get the correct result set, and all is well.

The query will continue to work perfectly well until the day arrives that the company decides to build a new factory as sales increase:

The extra row in the factories table causes our query to generate an error now:

The error is telling us that the outer query expected a scalar value, but our subquery returned a result set. We can fix the issue and list all factories that manufacture each product by using a JOIN:

One More Thing...

Be aware that the same error can occur in any clause where a column or expression is tested against a subquery, for example "column = (SELECT value FROM Table)". In that case, the solution is to use the IN() function instead of the equality (=) operator.

Outer Joins and Cartesian Products

In this series on Top SQL Query Mistakes, we've been exploring how seemingly intuitive ways of constructing SQL queries can result in anti-patterns that lead to erroneous results and/or performance degradation. Last week, we took a break from the series to talk about Predicates in SQL. In this installment, we'll be learning how their placement can adversely affect query execution - particularly in outer joins.

What Are Outer Joins?

There are four basic join types employed in linking related tables and views: inner, left, right, and outer. With an inner join, rows from either table that are unmatched in the other table are not returned. In an outer join, unmatched rows in one or both tables can be returned. The last three join types are all instances of outer joins, whereby:

• LEFT JOIN returns only unmatched rows from the left table.
• RIGHT JOIN returns only unmatched rows from the right table.
• FULL OUTER JOIN returns unmatched rows from both tables.

How Outer Joins Go Wrong

While outer joins certainly have their place in the database practitioner's arsenal, developers have a tendency of using them even in situations where they are not needed. Moreover, an outer join query can produce completely different results depending on how you construct it, and where you place predicates in your query. To illustrate, let's look at an example.

We would like to retrieve a list of ALL customers (whether they placed any orders or not), along with the total number of orders that they placed since the beginning of June, 2005. Do do so, we would employ an outer join to link the customers and orders tables as follows:

SELECT C.customerName, count(O.customerNumber) AS 2005_ordersFROM customers AS CLEFT OUTER JOIN orders AS O  ON C.customerNumber = O.customerNumberWHERE O.orderDate >= '2005-05-01'GROUP BY C.customerNameORDER BY 2005_orders DESC;

The result should contain every possible combination of rows from the first and second table, also known as a Cartesian product. Unfortunately, when we run the query in Navicat Premium 16, only 13 rows are returned, even though there are 122 unique customers in the table (not shown):

To understand where we went wrong, let's rebuild the query one step at a time, starting with only the columns and outer join:

Now we are getting all of the customers. Those who have not placed any orders have NULL customerNumbers, since they are coming from the orders table.

Now, let's apply the WHERE clause predicate:

All of sudden, we've lost many customers! The problem is that the predicate in the WHERE clause turned the outer join into an inner join.

To correct the issue, we need to add the WHERE predicate to the join condition:

We can now adjust our original query to fetch all customers:

The Moral of the Story

Always be careful of where you filter out rows. In the above example, the WHERE clause was the issue; in a more complex example, where multiple joins occur, the incorrect filtering may happen on a subsequent table operator (like join to another table) instead in the WHERE clause.

This week, we're going to briefly hit the Pause button from the Some Top SQL Query Mistakes series in order to talk about Predicates in SQL. The reason is that Predicates will factor into Part 3 of the Top SQL Query Mistakes series.

What Is a Predicate?

A predicate is simply an expression that evaluates to TRUE, FALSE, or UNKNOWN. Predicates are typically employed in the search condition of WHERE and HAVING clauses, the join conditions of FROM clauses, as well as any other part of a query where a boolean value is required.

There are numerous types of predicates, including:

• Comparison
• LIKE
• BETWEEN
• IN
• EXISTS
• IS NULL (/INTEGER/DECIMAL/FLOAT...)

In the remainder of this article, we'll examine a few examples of the above Predicate types.

Comparison Predicates

Any time that we use a comparison operator in an expression, such as WHERE employee_salary > 100000, we are constructing a Predicate that evaluates to TRUE, FALSE, or UNKNOWN. Comparison operators include:

• =Equal to
• >Greater than
• < Less than
• >= Greater than or equal to
• <= Less than or equal to
• <> Not equal to

Hence, a comparison predicate takes the form of:

expression_1 comparison_operator expression_2

In a comparison predicate, expression2 can also be a subquery. If the subquery does not return any rows, the comparison predicate evaluates to FALSE.

LIKE Predicate

IN SQL, the number one pattern-matching predicate is the LIKE operator, as it compares column values with a specified pattern. Like works with any character or date data type. Here's an example:

BETWEEN Predicate

The BETWEEN operator specifies a range, which determines the lower and upper bounds of qualifying values. For instance, in the Predicate income BETWEEN 5000 AND 20000 the selected data is a range of greater then or equal to 5000 and less then or equal to 20000. The Between operator can be used with numeric, text and date data types. Here's an example:

IN Predicate

An IN operator allows the specification of two or more expressions to be used for a query search. The result of the condition is TRUE if the value of the corresponding column equals one of the expressions specified by the IN predicate:

EXISTS Predicate

The EXISTS Predicate accepts a subquery as an argument. It returns TRUE if the subquery returns one or more rows, and returns FALSE if it returns zero rows.

Here's an example:

IS NULL Predicate

Use IS NULL to determine whether an expression is null, because you cannot test for null by using the = comparison operator. When applied to row value expressions, all elements must test the same.

The IS NULL predicate takes the following form:

IS [NOT] NULL

For example, the expression x IS NULL is TRUE if x is a null.

IS UNKNOWN is a synonym for IS NULL when the expression is of the BOOLEAN type.

Here's a query that uses the IS NOT NULL Predicate to fetch all actors whose last name is a non-NULL value:

Conclusion

In this blog, we interrupted the regularly scheduled blog to bring you this important lesson on SQL Predicates. Typically employed in the search condition of WHERE and HAVING clauses, the join conditions of FROM clauses, Predicates are expressions that evaluate to TRUE, FALSE, or UNKNOWN. We'll be seeing Predicates again in next weeks continuation of the Top SQL Query Mistakes series.

Part 2: Non-SARGable Query Conditions

Like most programmers, database developers tend to write code that is more or less a direct translation of a given request. The fact that most programming languages - SQL included - are designed to be human readable, also contributes to this problem. Why is this a concern? All programming languages execute certain operations faster than others. In relational databases, the query optimizer analyzes SQL queries and determines the efficient execution mechanisms called query plans. The optimizer generates one or more query plans for each query, each of which represent one possible way to run a query. The most efficient query plan is then selected and utilized to run the query. As it turns out, SQL that mimics the language of a request is seldom the most efficient approach.

In this installment of the Top SQL Query Mistakes series, we'll explore one example of a poorly written SQL statement and rewrite it in a way that increases efficiency.

Passing Indexed Columns to Functions

One faux pas that comes up over and over again in database developers' code is the passing of index columns to functions. To illustrate, let's execute a query against this table, which has an index on the varchar customerName column:

When asked to retrieve all customers whose name starts with the letter "R", one might be inclined to use the LEFT() function to return the first character of the customerName column:

Unfortunately, by passing the indexed customerName column to a function, the query engine must evaluate its result for every row in the table!

SARGable vs. Non-SARGable Queries

In relational databases, there is a term that is derived from a contraction of Search ARGument ABLE, aka, SARGable. A condition (or predicate) in a query is said to be SARGable if the DBMS engine can take advantage of an index to speed up the execution of the query. On the other side of the coin, a query that fails to be SARGable is known as a non-SARGable query. The effect is similar to searching for a specific term in a book that has no index, beginning at page one each time, instead of jumping to a list of specific pages identified in an index. Obviously, this has a negative effect on query time, so one of the steps in query optimization is to convert such conditions to be SARGable.

To make a condition such as the one above into a SARGable one, we need to avoid the use of functions on the indexed columns. To do that, we must express the request with this logically equivalent (and SARGable) query, using the Like operator:

Notice the greatly improved execution time.

Conclusion

In this second installment on Top SQL Query Mistakes, we learned how non-SARGable query conditions can degrade query performance by forcing the database engine to evaluate every row of a table. The fix is to express the request with a logically equivalent (and SARGable) condition that does not rely on a function call.

If you'd like to give Navicat 16 for MySQL a test drive, you can download a 14 day trial here.

NOT IN Versus NOT EXISTS

There's a term the is commonly thrown around in programming circles called "anti-patterns". It refers to a response to a recurring problem that is not only ineffective, but also risks being highly counterproductive. The term was originally coined in 1995 by computer programmer Andrew Koenig, in his book Design Patterns, as the antithesis of design patterns that are considered to be both reliable and effective.

Although SQL is not really a programmatic language, it turns out that it's equally susceptible to anti-patterns, especially when the query in question is fairly complex. Sometimes mistakes are hard to spot, and do not reveal themselves until the query is thrust into the pressure cooker of the production environment.

With the goal of catching SQL mistakes earlier, the next several blogs will be devoted to highlighting some of the most common culprits. We'll be using MySQL to execute today's examples, but the concepts are equally valid in any flavor of SQL.

NOT IN Versus NOT EXISTS

One common type of SELECT query retrieves data that is not included in a list of values. To illustrate, here are two very simple tables created in Navicat for MySQL 16. The first table contains colors:

The second table contains products:

What we would like to do is select all of the colors that have not yet been associated to any products. In other words, we need to construct a query that returns only those colors for which there is no product with that color. One might be tempted to employ the NOT IN predicate to fetch the records in question.

We would expect the following query to return two rows (for "black" and "green") when, in fact, an empty result set is returned:

The problem? The present of a NULL value in the color column on the products table, which are translated by the NOT IN predicate to:

color NOT IN (Red, Blue, NULL)

OR

NOT(color=Red OR color=Blue OR color=NULL)

The expression "color=NULL" evaluates to UNKNOWN and, what many database developers overlook, is that NOT UNKNOWN also evaluates to UNKNOWN! As a result, all rows are filtered out and the query returns zero rows.

This issue can also surface if requirements change, and a non-nullable column is updated to allow NULLs. Hence, even if a column disallows NULLs in the initial design, you should make sure your queries will continue to work correctly with NULLs, should things ever change.

The simplest solution is to use the EXISTS predicate instead of IN:

Problem solved!

So why does this work? Whereas the IN keyword will compare all values in the corresponding subquery column, EXISTS evaluates true or false. Consequently, using the IN operator, the SQL engine will scan all records fetched from the inner query. On the other hand, if we are using EXISTS, the SQL engine will stop the scanning process as soon as it found a match.

Conclusion

In this first installment on Top SQL Query Mistakes, we learned about how anti-patterns can occur in SELECT queries, starting with the erroneous use of the NOT IN predicate.

If you'd like to give Navicat 16 for MySQL a test drive, you can download a 14 day trial here.

Querying by Date

In this final installment in this series on Dates and Times in MySQL, we'll be putting everything we've learned thus far into practice by writing SELECT queries to obtain date-related insights into our data.

Selecting a Date from a Datetime Column

One of the first challenges database practitioners encounter when trying to query with dates is that a good deal of temporal data is stored as DateTime and Timestamp data types. For example, the Sakila Sample Database stores the customer table's create_date column as a Datetime:

Hence, if we try to select customer records that were created on a specific date, we can't simply supply a date value:

One simple workaround is to convert the Datetime values to Dates by using the DATE() function:

Now any record whose date matches ours will be returned.

Obtaining the Difference Between Two Dates

It is extremely common to perform queries that determine how long ago something happened. In MySQL, the way to do that is to employ the DATEDIFF() function. It accepts two date values and returns the number of days between them. Here's a simple example using Navicat for MySQL 16:

Notice that, in the above example, DATEDIFF() is telling us that the first date is 10 days later than the second one. We can also use an earlier date for the first argument and it will return a negative value:

Calculating Periods Other than Days

For periods other than days, we need to do a little conversion. For example, we can divide by 7 to obtain the number of weeks between two dates. Rounding is also employed to show whole weeks in the results:

ROUND(DATEDIFF(end_date, start_date)/7, 0) AS weeksout

For other time periods, the TIMESTAMPDIFF() function may be of help. It accepts two TIMESTAMP or DATETIME values (DATE values will auto-convert in MySQL) as well as the unit of time we want to base the difference on. For instance, we can specify MONTH as the unit in the first parameter:

SELECT TIMESTAMPDIFF(MONTH, '2012-05-05', '2012-06-04')-- Outputs: 0SELECT TIMESTAMPDIFF(MONTH, '2012-05-05', '2012-06-05')-- Outputs: 1SELECT TIMESTAMPDIFF(MONTH, '2012-05-05', '2012-06-15')-- Outputs: 1SELECT TIMESTAMPDIFF(MONTH, '2012-05-05', '2012-12-16')-- Outputs: 7

A More Complex Example

Once you've got the hang of the DATEDIFF() function, you can using it in more advanced ways. Case in point, here's a query that uses the DATEDIFF() function to calculate the average number of days that customers keep their film rentals before returning them:

To do that, the results of the DATEDIFF() function is passed to the AVG() function and then rounded to 1 decimal place.

Series Conclusion

We've covered a lot of ground in this series on Dates and Times, including:

• MySQL's five temporal data types
• some important date/time-oriented functions
• how to create dates and times in MySQL
• querying by date

While there certainly is a lot more to working with temporal data in MySQL, hopefully this series gave you a good head start on your road to MySQL proficiency.

Date/Time Creation

In this this series on Dates and Times, we've explored MySQL's five temporal data types, as well as some of its many date/time-oriented functions. In this installment, we'll be covering a few ways to create dates and times in MySQL.

Using the MAKEDATE() Function

In Part 3, we took a brief look at the MAKEDATE() function. It takes a year and dayofyear and returns the resulting Date value. For instance, MAKEDATE(2021, 200) would return a Date of "2021-07-19". The downside to this function should be readily obvious; it takes some calculation to determine the dayofyear if you have a year, month, and day. In that case, you can make a DATE by combining MAKEDATE() with DATE_ADD(). MAKEDATE() with a day of 1 will give you a DATE for the first day of the given year, and then you can add to it the month and day with DATE_ADD(). Here's an example that sets the year and month only:

This SELECT statement includes the day as well:

The MAKETIME() Function

If you're looking to create a TIME only, MAKETIME() returns a time value calculated from the hour, minute, and second arguments:

The second argument can have a fractional part for milliseconds:

The STR_TO_DATE() Function

Another option for creating a DATE, TIME, or DATETIME is to use the STR_TO_DATE() function. It takes a date string and a format string and returns:

• a DATE value if the string contains only date
• a TIME value if the string contains only time
• a DATETIME value if the format string contains both date and time parts

Moreover, if the date, time, or datetime value extracted from str is invalid, STR_TO_DATE() returns NULL and produces a warning.

Some Examples

Here are a couple of Dates in Navicat for MySQL 16:

Scanning starts at the beginning of str and fails if format is found not to match. Meanwhile, extra characters at the end of str are ignored:

Unspecified date or time parts have a value of 0, so incompletely specified values in the date/time string produce a result with some or all parts set to 0:

For the full list of specifiers that can be used in format, see the DATE_FORMAT() function description in the official MySQL docs.

Combining the MAKEDATE(), MAKETIME(), AND STR_TO_DATE() Functions

If we had two separate DATE and TIME values, we could get a DATETIME value by concatenating the the results of MAKEDATE() and MAKETIME() and then passing the combined string to STR_TO_DATE(). While that might sound like a lot of work, it's really quite simple in practice:

Conclusion

In this installment of the Working with Dates and Times in MySQL series, we covered a few ways to create dates and times in MySQL using some of its specialized date and time functions. In the next installment, we'll look at how to use temporal data in your SELECT queries.

Important Functions

In the first two installments of this series on Dates and Times, we covered MySQL's five temporal data types. Now it's time to turn our attention to some of MySQL's many date/time-oriented functions.

Getting the Current Date and Time

Back in May of 2021, we covered some of SQL Server's notable Date & Time functions, starting with how to obtain the current date and time. It offers the GETDATE() function for that purpose. MySQL's equivalent function is simply called NOW(). In Navicat for MySQL 16, we can invoke this function without connecting to a database, since we aren't selecting any table columns:

As mentioned in Part2, the TIMESTAMP type is similar to DATETIME, but are generally used to track changes to records. To obtain the current date and time as a TIMESTAMP, we can use the current_timestamp() function. Here's its output:

Getting the Current Date Without the Time

If you only want to get current date in MySQL, you can use either the curdate() or current_date() functions. The system variable current_date also works. In any event, all three give latest date in YYYY-MM-DD format:

Getting the Current Time Only

Likewise, we can get the current time in MySQL using the curtime() or current_time() functions, as well as the current_time system variable. These all give the latest time in HH:MM:SS format:

Parsing Out Individual Date Parts

SQL Server offers the versatile DATEPART() function to extract part of a datetime. MySQL provides the equivalent EXTRACT() function for this purpose. Similar to the SQL Server function, EXTRACT() accepts a part unit and the date:

EXTRACT(part FROM date)

Here are all of the valid part values:

• MICROSECOND
• SECOND
• MINUTE
• HOUR
• DAY
• WEEK
• MONTH
• QUARTER
• YEAR
• SECOND_MICROSECOND
• MINUTE_MICROSECOND
• MINUTE_SECOND
• HOUR_MICROSECOND
• HOUR_SECOND
• HOUR_MINUTE
• DAY_MICROSECOND
• DAY_SECOND
• DAY_MINUTE
• DAY_HOUR
• YEAR_MONTH

Being February at the time of this writing, the following call to EXTRACT() yields a value of "2":

As the following query shows, it is currently 43 minutes past the hour:

Additional Date Parsing Functions

Having trouble remembering all of the part units? That's OK, because MySQL provides separate functions for date and time parsing a well.

For parsing either the date or time from a datetime value, there are the DATE() and TIME() functions, respectively:

To split a date into its constituent parts, we can use the YEAR(), MONTH(), and DAYOFMONTH() (or DAY()) functions:

Time portions also get their own function: HOUR(), MINUTE(), and SECOND() respectively:

Constructing a Datetime From Separate Parts

In MySQL, there are many ways to create a datetime from separate date and time parts, enough to garner their own article. For now, let's look at one way to create a Date. The MAKEDATE() function returns a date given a year and dayofyear. Here's an example:

Going Forward

In this blog, we explored some of MySQL's many date/time-oriented functions. In the next installment, we'll cover some other ways to create dates and times in MySQL.

TIMESTAMP and YEAR Types

Welcome back to this series on working with dates and times in MySQL. In the first two installments, we're looking at MySQL's temporal data types. Part 1 covered the DATE, TIME, and DATETIME data types, while this installment will cover the remaining TIMESTAMP and YEAR types.

The TIMESTAMP Type

The TIMESTAMP type is similar to DATETIME in MySQL in that both are temporal data types that hold a combination of date and time. This begs the question why have two types for the same information? For starters, timestamps in MySQL are generally used to track changes to records, and are often updated every time the record is changed, whereas datetimes are used to store a specific temporal values. Another way to think about it is that DATETIME represents a date (as found in a calendar) and a time (as seen on a wall clock), while TIMESTAMP represents a well defined point in time. This distinction could be very important if your application handles timezones, as how long ago was '2009-11-01 14:35:00' depends on what timezone you're in. Meanwhile, 1248761460 seconds since '1970-01-01 00:00:00 UTC' always refers to the same point in time.

In terms of storage, a TIMESTAMP requires 4 bytes while DATETIME requires 5. TIMESTAMP columns store 14 characters, but you can display it in different ways, depending on how you define it. For example, if you define the column as TIMESTAMP(2), only the two-digit year will be displayed (even though the full value is stored). The advantage to this approach is that, if you later decide to display the full value, you can change the table definition, and the full value will appear.

Below is a list of various ways to define a TIMESTAMP, and the resultant display format:

• TIMESTAMP(14): YYYY-MM-DD HH:MM:SS
• TIMESTAMP(12): YY-MM-DD HH:MM:SS
• TIMESTAMP(10): YY-MM-DD HH:MM
• TIMESTAMP(8): YYYY-MM-DD
• TIMESTAMP(6): YY-MM-DD
• TIMESTAMP(4): YY-MM
• TIMESTAMP(2): YY

In the Navicat 16 Table Designer, a timestamp's precision may be defined in the Length column:

If no Length is supplied, as in the above example, Navicat displays the full field, as if it was declared as TIMESTAMP(14):

The YEAR Type

Many DBAs opt to store years as integers. While that can certainly work, it is more efficient to use MySQL's dedicate YEAR type for that purpose, as the YEAR type uses a mere 1 byte. It can be declared as YEAR(2) or YEAR(4) to specify a display width of two or four characters. If no width is given the default is four characters. YEAR(4) and YEAR(2) have different display formats but have the same range of values:

• For 4-digit format, MySQL displays YEAR values in YYYY format, with a range of 1901 to 2155, or 0000.
• For 2-digit format, MySQL displays only the last two (least significant) digits; for example, 70 (1970 or 2070) or 69 (2069).

Here's an example of a year column in the Navicat Table Designer with a four digit format:

As a result, we see the full year in the table:

Conclusion

That concludes our exploration of the five MySQL temporal data types. The next installment will cover some useful date and time functions.

DATE, TIME, and DATETIME Types

The vast majority of databases store a great deal of "temporal" data. Temporal data is simply data that represents a state in time. An organization may collect temporal data for a variety of reasons, such as to analyze weather patterns and other environmental variables, monitor traffic conditions, study demographic trends, etc. Businesses also routinely need to store temporal data about when orders were placed, stock refilled, staff hired, and a whole host of other information about their day-to-day business.

You may be surprised to learn that relational databases do not store dates and times in the same way. MySQL is especially prescriptive. For instance, It stores date values using the universal yyyy-mm-dd format. This format is fixed and may not be changed. You may prefer to use a mm-dd-yyyy format, but it is not possible to do so. However, you can use the DATE_FORMAT function to format the date the way you want in the presentation layer, usually an application. In the first two installments on working with Dates and Times in MySQL, we'll be looking at MySQL's temporal data types, starting with DATE, TIME, and DATETIME.

Types At a Glance

MySQL provides five types for storing dates and times, some just for dates, others for time, and some that include both. Here's a table that summarizes each type:

The rest of this article will cover the first three types in more detail, while the next one will focus on the other two.

TheDATE Type

MySQL uses 3 bytes to store a DATE value. The DATE values range from 1000-01-01 to 9999-12-31. Moreover, when strict mode is disabled, MySQL converts any invalid date e.g., 2015-02-30 to the zero date value 0000-00-00. In Navicat 16, you can select the DATE type in the Table Designer from the Types drop-down:

To set a DATE value, you can simply choose it using the calendar control:

Of course, you can also insert a DATE using the INSERT statement:

TheTIME Type

MySQL uses the 'HH:MM:SS' format for querying and displaying a time value that represents a time of day, which is within 24 hours. To represent a time interval between two events, MySQL uses the 'HHH:MM:SS' format, which is larger than 24 hours.

Here is the TIME type in the Navicat 16 Types drop-down:

To set a TIME value, Navicat provides the TIME INPUT control:

Here's an INSERT statement that sets a start and end time:

TheDATETIME Type

Quite often, you'll need to store both a date and time. To do that, you can use the MySQL DATETIME type. By default, DATETIME values range from 1000-01-01 00:00:00 to 9999-12-31 23:59:59. When you query data from a DATETIME column, MySQL displays the DATETIME value in the same YYYY-MM-DD HH:MM:SS format.

A DATETIME value uses 5 bytes for storage. In addition, a DATETIME value can include a trailing fractional second up to microseconds with the format YYYY-MM-DD HH:MM:SS[.fraction], for example, 2015-12-20 10:01:00.999999.

For inputting DATETIME values, Navicat provides the DATETIME INPUT control, which combines the DATE and TIME controls:

DATETIME values may be set using a string literal that contains the "T" time portion delineator or by casting to a DATETIME:

Going Forward

Having explored the DATE, TIME, and DATETIME Types, the next installment will cover the remaining two temporal types: TIMESTAMP and YEAR.

Back in May of 2021, we examined a few of SQL Server's Important SQL Server Functions. Now, it's time to turn our attention to MySQL to see what it offers us in terms of math and numeric functions. To see how they work in practice, we'll use them in queries that we'll run in Navicat 16 for MySQL.

Comparing SQL Server and MySQL Functions

In the Important SQL Server Functions article, we reviewed the Abs, Round, Ceiling, and Floor functions. As it turns out, not only does MySQL also implement these functions, but with exactly the same names. Perhaps this is not surprising, as these are fundamental numeric functions across database products and programming languages.

Since the functions are the same in both DBMSes, there's no point in rehashing their use. Instead, we'll forge onwards and explore other useful numeric functions in MySQL.

AVG

You probably already know what the Average, or Arithmetic Mean, is. You may even know that it is calculated by adding all of the value within a data set, and then dividing that result by the number of data points in the set. Hence, if we had five numbers such as 4, 5, 6, 5, 3, we would calculate their average as follows:

(4 + 5 + 6 + 5 + 3) / 5 = 4.6

Simple enough to do with a small sample, but what happens when you have 10,000 rows? The answer, or course, is to use MySQL's built-in AVG function (identically named in SQL Server, by the way). All we need to do is provide it with a numeric expression and it returns its average value. Here's its simple syntax:

AVG(expression)

Most of the time, you'll find yourself passing in a column name whose average you'd like to calculate. For example, here's a query that gives us the average running time of all the films in the Sakila Sample Database:

The GROUP BY breaks up values by the category_id so that averages are based on each Film Type, i.e., "Action", "Drama", etc.

By passing the results of the AVG function to ROUND, we can omit some of the extra decimal places.

A More Complex Example

The interesting thing about numeric functions is that they can be used as part of larger calculations. Case in point, here's a query that shows how many film categories in which the average difference between the film replacement cost and the rental rate larger than 17 dollars:

To calculate the replacement cost, the average rental rate is subtracted from the average replacement cost. No need for temporary variables; just subtract the results of one function from the other:

( AVG( replacement_cost ) - AVG( rental_rate ) ) AS replace_sub_rental

MIN/MAX

Have you noticed that a lot of numeric functions have three letter names? Not sure why that is, but here are two related functions for calculating the minimum and maximum values of a set. Again, the most typical usage is to pass a column name to the function. The following query selects film details for the first and last rentals, according to the rental_date column. As such, it is passed to both the MIN and MAX functions:

Mixing aggregate functions and scalar data can be problematic, so the MIN and MAX rental_dates are fetched within a subquery for comparison to those of each Film table row.

Conclusion

This blog presented a few useful numeric functions in MySQL, including AVG, MIN, and MAX using Navicat 16 for MySQL as our database client. Speaking of which, if you'd like to give Navicat 16 for MySQL a test drive, you can download a 14 day trial here.

One of the key ingredients to writing effective SQL queries is the ability to articulate a wide variety of conditions using SQL syntax. One condition that gives both newbies and experienced database developers pause for thought is the Exclusive OR. Software programmers tend to be more familiar with the syntax for the Exclusive OR condition, probably because most programming languages support the XOR logical operator, whereas many databases do not.

In simple terms, the Exclusive OR condition is similar to the regular OR, except that, in the case of the Exclusive OR, only one of the compared operands may be true, and not both. In this blog, we'll learn how to express an Exclusive OR condition for a variety of databases, whether they support the XOR operator, or not.

Using the XOR Operator

Some popular relational databases, such as MySQL, support the XOR operator, making writing Exclusive OR conditions fairly trivial. To illustrate, let's consider a scenario where we need to find customers that reside within a specific city, or whose account was created after a specific date, but not both. More specifically, say that we want to see customers who reside in Lethbridge, Alberta, OR, if they do not reside in Lethbridge, whose account was created after Jan 1st, 2020.

Here's just such a query, executed against the Sakila Sample Database using Navicat Premium 16:

Looking at the results, we can see that the first customer, whose account was created on 2020-07-07, has a store_id of 2, while the rest of the customers, all have a store_id of 1 (the Lethbridge store).

Meanwhile, if we replace the XOR to a regular OR, we now see customers who shop at store #1 whose accounts were also created after 2020-01-01:

Allowing both operands to evaluate to TRUE is what differentiates OR from XOR.

Writing an Exclusive OR Condition Where XOR Is Not Supported

Luckily, it's not that hard to formulate an Exclusive OR condition without the XOR operator; you just need to think about it a bit more. Mathematically speaking, x XOR y is equal to:

(x AND (NOT y)) OR ((NOT x) AND y)

We can simplify the above formula to the following for the purposes of SQL writing:

(A OR B) AND NOT (A AND B)

We'll try out this formula by rewriting the first query for SQL Server. If we try to execute it against that database, we get the following error, which states that SQL Server did not recognize the XOR operator:

Using the above formula, we can rewrite the XOR condition as:

WHERE   (ci.city = 'Lethbridge' OR c.create_date  > '2020-01-01')AND NOT (ci.city = 'Lethbridge' AND c.create_date > '2020-01-01')

Here are the results in SQL Server (note that the data in both databases is not identical):

Conclusion

In today's blog, we learned how to articulate an Exclusive OR condition in a variety of databases, both using the XOR operator, and without.

If you'd like to give Navicat 16 a test drive, you can download a 14 day trial here.

Recently, we learned how to generate test data using Navicat 16's new Data Generation tool. It can help produce a large volume of complex testing data over multiple related tables, all guided by a multi-step wizard. In today's follow-up, we'll go through the process of creating a MySQL test database - all using Navicat 16.

Duplicating a Production Database

For optimal testing, the best approach is to duplicate the structure of your production databases (DBs), while replacing the "real" data with sanitized test values. For the purposes of this tutorial, we'll use an instance of the MySQL classicmodels Sample Database as the source that is the basis for our test database. Here it is in Navicat Premium 16:

There is usually more than one way to accomplish a task in Navicat. Duplicating a database is no exception. There are several ways to do it; here are a couple:

Create a New Database

Rather than copy the database, we can create a brand new one and then generate the test data for it. To do that:

• In the Navigation pane, right-click your connection and select New Database:

  

• Enter the database properties in the pop-up window:

  

  Hint: if you aren't sure what Character Set and Collation to use, you can open the Edit Database dialog on the source DB to see their values.
• To copy over the table structures without data, simply select all of the tables in the Objects pane and drag them over to the new DB. A popup menu will appear asking you whether to copy over the Structure and Data or the Structure only:

  

  Choose the latter option.

Generate Tables From Model

Most organizations maintain model diagrams of their databases. Navicat's Modeling tool can generate database objects from a model (forward engineering) as well as generate a model from an existing DB (reverse engineering). Let's use it now to generate our test tables.

• Follow the two first steps from the last exercise to create the classicmodels_test database.
• Click the Model button on the main toolbar to see available models:

  

  Hint: if you don't have a model for your database, you can generate one by right-clicking the database in the Navigation Pane and choosing Reverse Schema to Model from the popup menu.
• Open the model in the Modeling tool by selecting the model in the Objects pane and clicking the Design Model button in the Objects pane toolbar:

  

• In the Modeling tool, select File -> Synchronize to Database... from the main menu.
• In the Synchronize to Database dialog, designate classicmodels_test as the target database and click the Compare button:

  

• Navicat will then determine which objects to create, update, or drop to synchronize both databases. In our case, it will generate all of the necessary tables:

  

• On the next screen, we can review the SQL statements that will be executed:

  

  Click the Start button to run the script.
• We'll get a detailed progress report as the script runs:

  

From there, we only need to follow the steps outlined in the Create a Model from a Database in Navicat blog.

Conclusion

Navicat 16 provides numerous options for duplicating an existing database, either on the same server or in a completely different environment.

Speaking of Navicat 16, if you'd like to give it a test drive, you can download a 14 day trial here.

As mentioned in the recent Present Your Data More Effectively with Navicat 16 blog, Navicat 16 added some new charting features, such as support for more data sources and chart types as well as an increased focus on usability and accessibility. These improvements coincide with the release of the new standalone Navicat Charts Creator. This blog will provide a tour of the Navicat Charts Creator and demonstrate how the Navicat Charts Creator can help you to gain deeper insights from your data.

About Workspaces

As the name suggests, a workspace is the place that gathers various resources together to work with as a cohesive unit. In the case of the Navicat Charts Creator, the workspace contains dashboards, charts and data sources. You can create multiple dashboards, charts and data sources within a single workspace. If you are new to the Navicat Charts Creator you can open the sample workspace and explore its contents:

Creating a Workspace

In the main window, click New Workspace. You're now ready to create data sources, charts and dashboards:

When it's time to save your workspace, you have the option of saving it locally or in the Cloud. To save locally, you would choose File -> Save from the main menu, while to save to the Cloud, you would select File -> Save to Cloud. From there, you only need to enter the workspace name and choose the project.

Data Sources

When building a chart, you will need to specify a data source that will supply the chart data via a dataset. The fields in the dataset can be used to construct a chart. Data sources can reference tables in your connections or data in files/ODBC sources, as well as select data from tables on different server types.

Creating a Data Source

To create a data source:

• In the Workspace window, click the New Data Source button, located directly under the main toolbar.
• Enter the name of the data source and select the desired connections, files or existing data sources. On the New Data Source dialog, you can choose from three options: Database, File/ODBC, or Connections in Existing Data Sources:

  

Upon clicking the OK button, a tab will open where you can edit the data source. Moreover, if you want to add more connections, you can click the plus (+) symbol at the top of the Connections pane and repeat the above steps.

To add tables to the Design pane, simply drag and drop them from the Connections pane! Likewise, nodes may also be connected to one another to create joins via drag and drop. Join types may be configured if necessary by clicking on them. To view the table data at any time, click the Preview button.

Charts

A chart is what provides a visual representation of the data in your data source. Mapping to a single data source, a chart can display correlations between several fields in the data. Navicat Charts Creator supports no less than 17 chart types! You can even make the chart interactive by adding a control chart. Here's one that allows the user to select order date months:

Conclusion

I hope that you enjoyed the tour of the Navicat Charts Creator. This blog really only scratched the surface of what it can do, but we'll be looking at the Navicat Charts Creator's capabilities in more detail over the next several weeks. In the meantime, you can download the Standalone Charts Creator and try it for free for 14 days!

There are many times where you'll want to see the relative contribution of a row (or group of rows) to the total row count. In other words, what percentage of the total count a row represents. To illustrate, let's take the following table, shown in Navicat Premium 16:

We can easily find out how many orders were received for each type of fruit by combining the count() function with the Group By clause:

So now, how would we view what percentage each fruit's orders contributed to the total number of orders? In fact, there are three standard ways to calculate row percentages in SQL. They are:

• Using OVER() clause
• Using subquery
• Using a Common Table Expression, or CTE

The rest of this blog will explore each of these in turn.

The OVER() Clause

Used predominantly with Window Functions, the OVER clause is used to determine which rows from the query are applied to the function, what order they are evaluated in by that function, and when the function's calculations should restart.

The OVER clause is the most efficient way to calculate row percentages in SQL, so it should be your first choice if efficiency is a priority for you. Here's the formula to obtain a percentage:

count(*) * 100.0 / sum(count(*)) over()

Adding the above SQL to our original query produces the following results:

Looks good, but some rounding wouldn't hurt. Unfortunately, that's not easily done using the over() clause. Perhaps the next option will be more to your liking.

Using a Subquery

Not all databases support the OVER() clause, so the subquery approach can be a very valuable fallback solution. It's sometimes referred to as the "universal solution" since it works in all databases. Another benefit of this approach is that it is also the easiest to incorporate with functions such as Round(). Here is what we'll need to add to our query:

count(*) * 100.0 / (select count(*) from <YourTable>)

And here is the universal solution in action:

Using a Common Table Expression (CTE)

The With common_table_expression clause specifies a temporary named result set, known as a common table expression (CTE). We can then select from the temporary result set to apply more functions to retrieved fields. IN our case, we can apply the sum() function to the counts to obtain the percentages:

Keep in mind that this approach is the least efficient as the CTE basically runs a second query against the results of the inner (initial) one. That being said, there may be times that you'll need to use a CTE to perform additional processing that you couldn't easily do in one go.

Conclusion

In this blog, we learned three ways to express the relative contribution of a row (or group of rows) to the total row count as a percentage. Each approach has its own strengths and weaknesses, so you'll have to choose one based on your specific requirements.

If you'd like to give Navicat 16 a try, you can download a 14-day fully functional FREE trial of Navicat here.

Throughout the past several weeks, we've been looking at Navicat 16's new features. While exciting to see, one should not discount the many improvements to Navicat that enhance its already great User Interface (UI) and workflow. Hence, today's blog will focus on improvements whose aim is to maximize performance and productivity.

Connection Profile

Keeping with the new work from home paradigm, Navicat 16 lets you configure multiple profiles for out-of-office users who may need to switch to a more secure connection depending on the location of the device they are using. To create a Connection Profile:

• In the connection window, click .

  

• Click + New Connection Profile -> New Profile.
• Enter the name of the profile.
• Enter the connection settings.
• Click OK.

Then, to switch profiles:

• In the main window, right-click a connection and select Switch Connection Profile.
• Select the profile name.

You can also set the default active profile in the connection window.

Value Picker

In the Table Viewer, you can filter records according to numerous criteria using the Filter tool. In Navicat 16, it has been fully rewritten to embed an intuitive panel for selecting values from a list, or inputting possible values to further limit the data in the exact way you want:

It even includes a Search field for looking up specific values:

Field Information

Also in the Table Viewer, the right-hand panel now includes field information that gives you a quick view of column characteristics. It can help you to get column information and compare columns with ease.

Details include:

• Data type
• Not null
• Default value
• Comments

Here's an example using the orders table of the classicmodels sample MySQL database:

Query Summary

Located at the bottom of the SQL Editor, the new query Summary tab shows a detailed summary of each SQL statement. It's an easy-to-read, one page summary of the health and performance of your queries that includes a link to jump over to the potential errors:

You can also view the full query by clicking on the ellipsis [...] button:

The Query Summary tab is especially useful in cases where you have many statements in the editor - especially Data Manipulation Language (DML) statements such as UPDATE, INSERT INTO, DELETE, and CREATE:

Conclusion

In this blog we explored four improvements in Navicat 16 whose aim is to maximize performance and productivity. These updates are all part of Navicat's commitment to maintaining your databases and ensuring that Navicat products perform at the highest possible level, so that your database administration and development activities are both efficient and fulfilling.

If you'd like to give Navicat 16 a try, you can download a 14-day fully functional FREE trial of Navicat here.

With all of the excitement surrounding the release of Navicat 16, other noteworthy developments have been overshadowed somewhat. Perhaps none more so that the new standalone Navicat Charts Creator. Charting has been a part of Navicat products for some time now. Navicat 15 went even further to include data visualization in order to help identify trends, patterns and outliers. Navicat 16 adds even more features, by supporting more data sources and chart types as well as a focus on usability and accessibility. Suffice to say, Navicat can deliver information and present your findings in dashboard for sharing to a wider audience than ever before. In today's blog, we'll take a quick tour of Navicat 16's new charting tools.

Data Connectors

In order to create visualizations of your data, you first need to connect to a data source. Navicat Charts Creator allows you to quickly and securely connect to any data source of your choice via four built-in data connectors. These include:

• Relational DB:For working with relational databases such as MySQL, MariaDB, PostgreSQL, Oracle, SQLite and SQL Server.

• File Types: Allows you to import data from external files such as Excel, Access, or CSV, from data stored on your computer, network, or accessible via an URL.

• ODBC:For importing data from any ODBC data source including Sybase, Snowflake and DB2.

• Linked File:Lets you link your chart to data in data sources to update the chart according to changes in the underlying data.

Chart Types

It's vitally important to choose the right type of chart so that your presentations convey the message you want to communicate. To that end, Navicat 16 includes a wide array of chart types, ranging from standard to exotic:

• Bar Chart
• Line / Area Chart
• Bar / Line Chart
• Pie Chart
• Heatmap / Treemap
• Pivot Table
• Scatter Chart
• Value
• Control
• KPI

Navicat 16 also introduces the Waterfall, Tornado, and Gauge chart types.

Dashboards