Application Metadata Analyzer

About the Endpoint

     As of the Qlik Sense June 2018 release, a new application level metadata endpoint is available via the Qlik Sense Proxy (exposed via the Engine). Data is populated for this endpoint post-reload in a June 2018+ environment. From this data, a Qlik application has been built that ingests it and provides lightweight dashboard to analyze your applications. The URL to access this data is from a single application is

http(s)://{server}/api/v1/apps/{GUID}/data/metadata

where server is a server that the application can be accessed over and GUID is the application ID. Note that the application does not need to be lifted into RAM for the metadata to be accessed.

Data from this endpoint includes:

• server metadata including number of cores, total RAM
• reload time
• app RAM base footprint
• field metadata including cardinality, tags, total count
• table metadata including fields, rows, key fields

Example output from that endpoint can be seen here.

Application Metadata Analyzer App

The Application Metadata Analyzer is a Qlik Sense application that collates the metadata from every application on your server to give you a single app view of your application metadata along with additional data from the QRS API. Adding this additional data allows us to visualize who owns what applications, are they published or private, what nodes are they available on, etc. That application can be found here, and is shipped with Qlik Sense Enterprise on Windows as of the September 2019 release: https://community.qlik.com/t5/Qlik-Monitoring-Administration/Sense-App-Metadata-Analyzer/gpm-p/1592163.

Data Connection Analyzer

About

The Qlik Sense Data Connection Analyzer is a Qlik application that parses script log files and queries the QRS API, allowing analysis of data connection usage, patterns, age, and cleanliness across a Qlik Sense site. In short, this app will help to run a leaner, more performant, and more easily and holistically governed Qlik Sense site.

The application is supported by Qlik's Americas Enterprise Architecture team, and uses only native connections and capabilities of the Qlik platform—making it plug-and-play.

The application can be found on GitHub here: https://github.com/eapowertools/qs-data-connection-analyzer

Common Questions & Associated Benefits of the App

1. Which connections are no longer used? (A data connection is found in a script log and exists in the Qlik Sense site, however no associated applications currently exist that at one point had used it).
   • Removing unused connections will increase performance across a site, as less connections will have to be evaluated in security rule evaluations. If a site has hundreds or thousands of connections, this calculation time can build up.
   • Removing unused connections makes general administration easier, as there is less to manage.
   
   
2. What connections have never been used? (A data connection exists in the Qlik Sense site, but no reference exists to it in any script log.)
• It is common that users will create data connections to test connectivity, but then never actually use them. By leveraging this app, one can identify connections that have never been used and have existed for x amount of time (say 90+ days), so that action can be taken to remove them. The benefits from both simpler management and performance are listed in the bullet above. What connections have been deleted that used to be used? (A data connection that is found in script logs, but no longer exists in the Qlik Sense site and no app is using it.)


3. What connections have been deleted that used to be used? (A data connection that is found in script logs, but no longer exists in the Qlik Sense site and no app is using it.)
• By parsing the script logs, one can visualize old data connection names/paths that can help to serve as an audit trail.


4. What are the most widely used data connections?
• Depending on how this application is deployed, whether it be strictly administrative or potentially visible to developers, this metric is important both administratively and socially throughout the organization.


5. Do we have duplicate data connections?
• By analyzing the connection strings, one can tackle duplicate connections to the same source data. This eases administration overhead and will ensure that there is reusability/consistency across the platform.


6. Who owns what data connections?
• This is valuable from an access/security perspective to ensure governance of data sources.


7. Who is using what data connections?
• While "User A" might own "Data Connection A", "User B" might also have read access to that data connection. This can of course be visualized through the audit capability of the QMC, however this application will physically reveal who is executing any reloads of those data connections, giving greater visibility and allowing a deeper level of auditing and governance.


8. Where are data connections being used?
• Let's say a data source is being transformed and will be moved from one database to location. The first question one might ask is, "What applications are using that connection, so we can re-route it to the new db and make adjustments to the load scripts?". This historically has not been easy to answer. This application allows one to select that connection and visualize apps that are using it.


9. Via what mechanism are data connections being used?
• The application visualizes what connections are being run as tasks, manually, or in ODAG (or other API) requests. This is crucial in understanding user behavior.

Screenshots