Description

Learn how the Store and Forward system works in Ignition with your database for data protection and efficient storage.

Video recorded using: Ignition 8.1

Transcript

(open in window)

[00:00] The store and forward system provides a reliable way for ignition to store data to the database. When Ignition attempts to run a query that would modify a database table in some way, the query will first enter the store and forward system. Once in the system, the records remain until we receive confirmation from the database that the query has been executed successfully. This allows Ignition to keep track of what records were entered successfully into the database and which records failed to forward, either due to an error in the connection or the data itself. We'll start off in the status section of my gateway here. I'll click on store and forward. Within the connection section, the screen presents me with some data trackers for the data points per second entering the system and which records errored out along with those that have been dropped. Additionally, there is this store and forward engine for my My SQL database. When creating a database connection in Ignition, we automatically create one of these store and forward engines for the connection to use.

[01:07] Earlier, I created some tags that are recording to a MYSQL database connection at a one second rate. Let's take a closer look at the store and forward engine to see how it's behaving. I'm going to click on the details button here. There are two areas the engines keep records, the memory buffer, and the local cache. When data is generated, the records will first be placed into the memory buffer. You can see that my memory buffer here is able to keep up with demand, so as historical records are being collected, they're entering the system and very quickly being shipped off to the database. If the records were not removed from the memory buffer in some time, or if a certain amount of data accumulates, it will be placed into the local cache. Now let's simulate a loss of connection. I'm going to shut down my database. Since Ignition can no longer communicate with the database, it will hold onto the records. You can see here that records are being moved over to the local cache.

[02:02] While I'm waiting for the local cache to grow a bit more, I wanted to mention that the store and forward system uses a first in first out method. So even when a connection is down like this, it will maintain the order the queries were submitted. This means if a process requires some queries run before others, the system will respect that. Now that we have some records in our local cache here, I'm going to start my database back up. I've started the database, but the store and forward system will still collect records until the database is fully up and running. While we're waiting for that, another benefit of the system is that Ignition can optimize the queries before sending them to the database. This results in a smaller impact on network bandwidth as opposed to directly writing to the database. Now that Ignition has detected the databases back up, it will start taking records from local cache and send them off to the database. And just like that, our local cache is empty. All my records have been safely inserted into my database.

You are editing this transcript.

Make any corrections to improve this transcript. We'll review any changes before posting them.