Mike  Kozey

Mike Kozey

1623748500

Create Your Own System Monitoring tool in Linux

Why create your own monitoring tools when there are so many better ones in the market? … Why not?

Maybe you want to do performance testing. Maybe you are using low spec system and want to create your own monitoring tools instead of inbuilt tools such as htop to reduce impact of analysis on data being analyzed. Maybe you just want to learn for the sake of learning. In this article I will help you write the script to monitor system. I will primarily use golang as it is fast, simple to can be compiled natively.

Of course, we will not all will be discussed all the method of monitoring all the parameters in this article. For code reference on what we are discussing here as well as some extra parameters, you can refer to my GitHub page here:  https://github.com/umangshrestha/system-monitoring

Before we start we need to learn little basics about pseudo files. We will try to keep the theory to minimum as possible.

Pseudo Files

Linux home folder structure.

In Linux, there are concepts called pseudo files. As the name suggests they are not a real files but look as such. They primarily reside in ram and store information about current system set. As such, they don’t pertain the data like other files after system restart. For system monitoring we mostly care only about /proc/ files. Learning the psudo files will take you one step closes to better understanding of linux.

/proc/cpuinfo -> nproc

nproc prints the number of CPU processing units available in the system. In normal programming this is the max number of threads that kernel allocates.

$ nproc
8

#linux #golang #go #shell

What is GEEK

Buddha Community

Create Your Own System Monitoring tool in Linux
Carmen  Grimes

Carmen Grimes

1598959140

How to Monitor Third Party API Integrations

Many enterprises and SaaS companies depend on a variety of external API integrations in order to build an awesome customer experience. Some integrations may outsource certain business functionality such as handling payments or search to companies like Stripe and Algolia. You may have integrated other partners which expand the functionality of your product offering, For example, if you want to add real-time alerts to an analytics tool, you might want to integrate the PagerDuty and Slack APIs into your application.

If you’re like most companies though, you’ll soon realize you’re integrating hundreds of different vendors and partners into your app. Any one of them could have performance or functional issues impacting your customer experience. Worst yet, the reliability of an integration may be less visible than your own APIs and backend. If the login functionality is broken, you’ll have many customers complaining they cannot log into your website. However, if your Slack integration is broken, only the customers who added Slack to their account will be impacted. On top of that, since the integration is asynchronous, your customers may not realize the integration is broken until after a few days when they haven’t received any alerts for some time.

How do you ensure your API integrations are reliable and high performing? After all, if you’re selling a feature real-time alerting, you’re alerts better well be real-time and have at least once guaranteed delivery. Dropping alerts because your Slack or PagerDuty integration is unacceptable from a customer experience perspective.

What to monitor

Latency

Specific API integrations that have an exceedingly high latency could be a signal that your integration is about to fail. Maybe your pagination scheme is incorrect or the vendor has not indexed your data in the best way for you to efficiently query.

Latency best practices

Average latency only tells you half the story. An API that consistently takes one second to complete is usually better than an API with high variance. For example if an API only takes 30 milliseconds on average, but 1 out of 10 API calls take up to five seconds, then you have high variance in your customer experience. This is makes it much harder to track down bugs and harder to handle in your customer experience. This is why 90th percentile and 95th percentiles are important to look at.

Reliability

Reliability is a key metric to monitor especially since your integrating APIs that you don’t have control over. What percent of API calls are failing? In order to track reliability, you should have a rigid definition on what constitutes a failure.

Reliability best practices

While any API call that has a response status code in the 4xx or 5xx family may be considered an error, you might have specific business cases where the API appears to successfully complete yet the API call should still be considered a failure. For example, a data API integration that returns no matches or no content consistently could be considered failing even though the status code is always 200 OK. Another API could be returning bogus or incomplete data. Data validation is critical for measuring where the data returned is correct and up to date.

Not every API provider and integration partner follows suggested status code mapping

Availability

While reliability is specific to errors and functional correctness, availability and uptime is a pure infrastructure metric that measures how often a service has an outage, even if temporary. Availability is usually measured as a percentage of uptime per year or number of 9’s.

AVAILABILITY %DOWNTIME PER YEARDOWNTIME PER MONTHDOWNTIME PER WEEKDOWNTIME PER DAY90% (“one nine”)36.53 days73.05 hours16.80 hours2.40 hours99% (“two nines”)3.65 days7.31 hours1.68 hours14.40 minutes99.9% (“three nines”)8.77 hours43.83 minutes10.08 minutes1.44 minutes99.99% (“four nines”)52.60 minutes4.38 minutes1.01 minutes8.64 seconds99.999% (“five nines”)5.26 minutes26.30 seconds6.05 seconds864.00 milliseconds99.9999% (“six nines”)31.56 seconds2.63 seconds604.80 milliseconds86.40 milliseconds99.99999% (“seven nines”)3.16 seconds262.98 milliseconds60.48 milliseconds8.64 milliseconds99.999999% (“eight nines”)315.58 milliseconds26.30 milliseconds6.05 milliseconds864.00 microseconds99.9999999% (“nine nines”)31.56 milliseconds2.63 milliseconds604.80 microseconds86.40 microseconds

Usage

Many API providers are priced on API usage. Even if the API is free, they most likely have some sort of rate limiting implemented on the API to ensure bad actors are not starving out good clients. This means tracking your API usage with each integration partner is critical to understand when your current usage is close to the plan limits or their rate limits.

Usage best practices

It’s recommended to tie usage back to your end-users even if the API integration is quite downstream from your customer experience. This enables measuring the direct ROI of specific integrations and finding trends. For example, let’s say your product is a CRM, and you are paying Clearbit $199 dollars a month to enrich up to 2,500 companies. That is a direct cost you have and is tied to your customer’s usage. If you have a free tier and they are using the most of your Clearbit quota, you may want to reconsider your pricing strategy. Potentially, Clearbit enrichment should be on the paid tiers only to reduce your own cost.

How to monitor API integrations

Monitoring API integrations seems like the correct remedy to stay on top of these issues. However, traditional Application Performance Monitoring (APM) tools like New Relic and AppDynamics focus more on monitoring the health of your own websites and infrastructure. This includes infrastructure metrics like memory usage and requests per minute along with application level health such as appdex scores and latency. Of course, if you’re consuming an API that’s running in someone else’s infrastructure, you can’t just ask your third-party providers to install an APM agent that you have access to. This means you need a way to monitor the third-party APIs indirectly or via some other instrumentation methodology.

#monitoring #api integration #api monitoring #monitoring and alerting #monitoring strategies #monitoring tools #api integrations #monitoring microservices

Ruth  Nabimanya

Ruth Nabimanya

1620633584

System Databases in SQL Server

Introduction

In SSMS, we many of may noticed System Databases under the Database Folder. But how many of us knows its purpose?. In this article lets discuss about the System Databases in SQL Server.

System Database

Fig. 1 System Databases

There are five system databases, these databases are created while installing SQL Server.

  • Master
  • Model
  • MSDB
  • Tempdb
  • Resource
Master
  • This database contains all the System level Information in SQL Server. The Information in form of Meta data.
  • Because of this master database, we are able to access the SQL Server (On premise SQL Server)
Model
  • This database is used as a template for new databases.
  • Whenever a new database is created, initially a copy of model database is what created as new database.
MSDB
  • This database is where a service called SQL Server Agent stores its data.
  • SQL server Agent is in charge of automation, which includes entities such as jobs, schedules, and alerts.
TempDB
  • The Tempdb is where SQL Server stores temporary data such as work tables, sort space, row versioning information and etc.
  • User can create their own version of temporary tables and those are stored in Tempdb.
  • But this database is destroyed and recreated every time when we restart the instance of SQL Server.
Resource
  • The resource database is a hidden, read only database that holds the definitions of all system objects.
  • When we query system object in a database, they appear to reside in the sys schema of the local database, but in actually their definitions reside in the resource db.

#sql server #master system database #model system database #msdb system database #sql server system databases #ssms #system database #system databases in sql server #tempdb system database

How to Install vnStat and vnStati to Monitor Network Traffic in Linux

VnStat is a console-based network traffic monitoring tool design for Linux and BSD. It will keep a log of the network traffic for selected network interfaces. To generate the logs, vnStat uses the information provided by the kernel.

In other words, it will not sniff the network traffic and will ensure the lite usage of the system resource. To use this software under Linux you will need at least version 2.2 of the kernel series.

Related ArticleVnStat PHP: A Web-Based Interface for Monitoring Network Bandwidth Usage

The latest version of vnStat 2.6 has been released on January 21, 2020, and includes several following features and fixes.

Features
  • Statistics remain available even after system reboots
  • Monitor multiple network interfaces at the same time
  • Multiple output options
  • Sort the data by hour, day, month, week or get the top 10 days
  • Generate png graphic of the output
  • Configure “Months” to follow up with different billing cycles you may have
  • Very light – consumes a really small portion of your system resources
  • Low CPU usage no matter how much traffic you generate
  • You don’t have to be root to use it
  • Select units dynamically (KB, MB, etc)
  • vnStati provides some new options like:
  • -nl / –nolegend (hides the rx/tx legend)
  • –altdate – use alternative date/time text location
  • –headertext – to customize the text in the image header.
  • You can generate output in image format.
  • Customizable options for content positioning and image background color to vnStat.cgi.
  • The interface bandwidth will be automatically detected.
  • Use JSON for output

In this article, we will show you how to install the vnStat and vnStati tool under Linux systems to monitor real-time network traffic.

#monitoring tools #networking commands #linux monitoring #vnstat #linux

Elton  Bogan

Elton Bogan

1600472100

HardInfo - Check Hardware Information in Linux

HardInfo (in short for “hardware information“) is a system profiler and benchmark graphical tool for Linux systems, that is able to gather information from both hardware and some software and organize it in an easy to use GUI tool.

HardInfo can show information about these components: CPU, GPU, Motherboard, RAM, Storage, Hard Disk, Printers, Benchmarks, Sound, Network, and USB as well as some system information like the distribution name, version, and Linux Kernel info.

Besides being able to print hardware information, HardInfo can also create an advanced report from the command-line or by clicking the “Generate Report” button in the GUI and saved in either HTML or plain text formats.

The difference between HardInfo and the other Linux hardware information tools is that the information is well arranged and easier to understand than other such tools.

#monitoring tools #open source #hardinfo #linux system information tool #linux

Remote Patient Monitoring Systems: Components, Types, Vendors & Implementation Process

Remote patient monitoring is the use of technology in healthcare organizations that enables real-time tracking of a patient. These software systems can easily find any sudden changes in a patient’s health and are also able to provide treatment plans. Remote patient monitoring is an important part of the e-health sector. Remote patient monitoring is also called telemonitoring. Remote patient monitoring solutions are here to stay, even after the coronavirus pandemic is gone. In this blog, we will explain how remote patient monitoring systems work, and about their different components. We will also look at the available options in the tech market and what to do before starting a remote patient monitoring program.

#remote patient monitoring solutions #remote patient monitoring integration #best patient monitoring systems #remote patient monitoring vendors #remote patient monitoring providers #mobile-apps