Announcements

Find and Grep utilities

bashThe powerful find and grep utilities on Unix/Linux : usage and practical examples. Beyond the basic cases, the examples address more complex needs with sometimes the power of regular expressions. A few examples covered : deleting files based on criteria, computing the size of images in a directory, finding invalid symbolic links. This paper is quite old (first version in 2002) but is enriched by the time with examples of interest to remember. Find and Grep utilities

awk, nawk and gawk utilities - Tutorial

bashThe awk, nawk and gawk (GNU Awk) utilities are very easy to use text file parsers. They handle efficiently text files of data delimited by a character. With an easy syntax to understand, operations for filtering rows, filtering columns, enriching content, converting formats, computing aggregates (averages, sums for example), etc. become child’s play with these utilities. awk, gawk and nawk differ only in a few very advanced features. awk is the utility to use without hesitation to parse very efficiently in seconds complex log files for example. Getting started with the nawk utility through examples, and contrary to common beliefs it is also available on Windows platforms. awk, nawk and gawk utilities - Tutorial

Google PageSpeed - LightHouse 6, new features CLS (Cumulative Layout Shift) and LCP (Largest Contentful Paint)

lighthouseWeb pages performance metrics are modified in LightHouse 6, they are more realistic and close to the user experience. A very interesting metric is introduced: CLS (Cumulative Layout Shift). The CLS statistics quantify the visual stability of web pages to avoid bad user experiences. All the tools are available with Chrome and LightHouse to diagnose the elements concerned in the visual instability of a page and in a few steps, the CLS score can be optimized, but everything will of course depend on the complexity of the pages and code. Google PageSpeed - LightHouse 6, new features CLS (Cumulative Layout Shift) and LCP (Largest Contentful Paint)

Mathematical expressions and equations in HTML with MathJax and AsciiMath

mathjaxMathML for writing mathematical expressions in HTML has not become a standard in HTML 5. Google Chrome does not support MathML, Google argued that powerful libraries in Javascript already exist and cover the needs for a very limited audience of scientists. Indeed, the MathJax javascript library coupled with AsciiMath is perfect when you want to write expressions in HTML pages with a professional and ergonomic rendering supported on all platforms. This article is not a MathJax tutorial, it introduces simple examples and especially how to dynamically load the MathJax.js library on demand using CSS classe names. A few words about the underscore character in AsciiMath syntaxes, the delimiter, the styles and the MathJax menu to quickly copy/paste. Mathematical expressions and equations in HTML with MathJax and AsciiMath

Measuring and storing pages performance metrics using Google PageSpeed Insights API and Python

lighthousePage speed is one of the key criteria in Google search engines ranking algorithms. The API Google PageSpeed Insights gives the ability to retrieve programmatically page speed LightHouse results executed on web pages from Google labs. With few Python code lines, the valuable statistics can be extracted and stored in a table for historization : useful when planning benchmarks before/after upgrades, updates. Measuring and storing pages performance metrics using Google PageSpeed Insights API and Python

Python, application configuration - environment variables, ini and YAML files

pythonObviously, no hard-coded application configuration in programs, Python programs or not. How to read (write) configuration data with Python from environment variables, INI files or YAML files? The os, configparser and PyYAML packages meet these needs. configparser for INI files and yaml for YAML files have their advantages and inconvenients if we venture to compare these 2 packages to make a choice. The JSON format is not discussed here, a dedicated article is published on this topic. Python, application configuration - environment variables, ini and YAML files

Python, handling dictionaries with dot notation

pythonPython is not Javascript. Habits, hard to remove them. How to use dot notation with Python for dictionaries ? It can be achieved natively using a class or SimpleNamespace but when dictionaries are nested, it becomes tedious without developing its own library. Python community packages exist to do the job : Prodict, python-box. Python, handling dictionaries with dot notation

Python, Managing HTTP requests with the packages requests and httplib2

pythonHTTP requests with the GET or POST method are widely used when exchanging data through programming API. With Python, 2 interesting packages for executing HTTP requests: requests and httplib2. The requests package is the easiest syntactically and natively includes a JSON decoder. About the httplib2 package, a very interesting feature: a caching mechanism to avoid unnecessary network calls in some application contexts using relatively static data. Python, Managing HTTP requests with the packages requests and httplib2

Python, Parsing arguments with the packages argparse and getopt

python2 native packages are available with Python to efficiently manage the command lines and arguments of a Python program: argparse and getopt. argparse is by far the most advanced package and offers the most productivity, about getopt a very strong resemblance to its habits when handling arguments in shell or in C language. Python, Parsing arguments with the packages argparse and getopt

Python, Reading and writing JSON with the package json

pythonFirst things to know when learning the Python language in the current technological context (data exchange, API…): how to read and write JSON data in a Python program ? Python integrates the json package natively for these operations with the load, loads, dump and dumps methods. Coming from the Javascript programming world, some habits handling Json data are not usable directly in Python (dot notation…). Python, Reading and writing JSON with the package json

Javascript - Listing active event listeners on a Web page

javascriptWhen developing a Web site, event listeners are implemented, by third party libraries or by your own. In some circumstances, when starting performance improvements tasks or when debugging behavior issues in event listeners due to a third party library, we may need to get an overview of all event listeners. How to get the full list : the events defined with addEventListener and the ones defined with the corresponding attribute ? It's not so trivial. Javascript - Listing active event listeners on a Web page

Standalone MariaDB ColumnStore 1.2.3 installation on Ubuntu - Getting started

mariadbIn order to discover the product, the installation of a standalone MariaDB ColumnStore system on Ubuntu is rather simple. Getting started is also quick with the combination of the mcsadmin and mcsmysql utilities. A technical starting point on this column-oriented database for real-time analytical processing of distributed and very high volume data. Standalone MariaDB ColumnStore 1.2.3 installation on Ubuntu - Getting started

How to list members of an Active directory group defined in Microsoft SQL Server

mssqlProject managers often directly ask the DBAs to provide the list of accounts belonging to a Microsoft Active Directory group defined in Microsoft SQL Server. Basic command lines exist (net group, net user) to list the members without having to disturb the administrators of the Active Directory domain controller (AD). How to list members of an Active directory group defined in Microsoft SQL Server

InfluxDB v2, Getting started. Preparing the migration from version 1.7

influxdbThe migration to InfluxDB v2 will need adjustments, many InfluxDB v1 functionalities are replaced in version 2. Migration procedure from version 1.x to version 2 is not yet unveiled, surely it will preconize export/import methods. The terminology is a little bit modified in version 2 : a database is now a bucket and an organization is necessarily attached to a bucket. No changes about the InfluxDB Line protocol. In this article : getting started with InfluxDB v2 and the breaks to manage when scheduling migration from version 1.x InfluQL is replaced by the language Flux, Continuous queries are replaced by tasks. Native support of the protocols Graphite, OpenTSDB… is removed, Telegraf will have to be used. Grafana plugin for Flux is not yet fully ready. InfluxDB v2, Getting started. Preparing the migration from version 1.7

Protecting SSH port with Fail2ban on Ubuntu - Setup and configuration

ubuntuOn Ubuntu servers, ufw (Uncomplicated Firewall) is a good tool to apply firewall rules on ports without having to use iptables commands which are quite difficult, but it's not enough to protect port 22 for SSH services. Fail2Ban is the complementary tool to ufw in order to protect an Ubuntu system from attacks on SSH. Fail2Ban is a Python package which interacts like ufw with iptables and can be installed in a Python virtual environment, this is the case in this article. Protecting SSH port with Fail2ban on Ubuntu - Setup and configuration

Understanding, installing and using the time series database InfluxDB 1.7

influxdbInfluxDB is a high performance time series database with efficient compression. The architecture is well designed (measurements retentions, shards) with easy ingestion, natively or through common time series protocols (OpenTSDB, Graphite…). Queries are SQL Like and the reporting is intuitive with Grafana or Chronograf. This introduction to InfluxDB v 1.7 also points out the breaks with the next major version 2.0. Understanding, installing and using the time series database InfluxDB 1.7

Python - Understanding and demistifying virtualenv

pythonWith Python, when installing a product, many dependencies can also be installed. It becomes tedious to manage polluting the Python global distribution : dependencies, package versions conflicts, binaries… The packages used by the most users can be installed in the system distribution and packages needed to only one user/product in a virtual environment using virtualenv. How to create and use virtual environments with virtualenv ? How packages and versions are managed in virtual and system distributions ? Python - Understanding and demistifying virtualenv

PostgreSQL - Planned Failover / Failback procedures with streaming replication

postgresqlMany articles deal with promoting a PostgreSQL standby server to primary server using the command pg_ctl promote. Many of them conclude the standby server shoud be be rebuilt when primary server is back. What about a scheduled maintenance on the primary server including shutdowns ? Do we have to rebuild the primary/standby system ? Obviously the answer is : NO. Assuming there is enough space disk for WAL files retention during the maintenance task, if the replication slot and the recovery option recovery_timeline_target are cautiously managed during the failover/failback procedures, there is no need to rebuild the whole standby system implying backup/restore databases and/or to use pg_rewind. PostgreSQL - Planned Failover / Failback procedures with streaming replication

Streaming replication with PostgreSQL 9.6, 10 and 11 - PostgreSQL Standby Databases

postgresqlImplementing a PostgreSQL 9.6 streaming replication on Linux Ubuntu 18.04 servers is very easy. In the streaming replication mode, the standby connects to the primary, which streams WAL records (Write Ahead Log) to the standby as they're generated, without waiting for the WAL file to be filled. Streaming replication allows a standby server to stay more up-to-date than is possible with file-based log shipping. The standby server can be in read only mode for reporting purposes for example. A replication slot ensures that the needed WAL files for the standby are not removed in the primary server before the standby server processes them. Multiple replication slots can be defined depending on the number of the standby servers. Streaming replication with PostgreSQL 9.6, 10 and 11 - PostgreSQL Standby Databases

PostgreSQL 9.6 - Quick setup and installation on Ubuntu 18.04

postgresqlA PostgreSQL 9.6 instance is installed and customized (port, database and configuration files…) on Ubuntu 18.04. A quick guide for creating a PostgreSQL database and managing this database with pg_ctl (start, stop, restart…) PostgreSQL 9.6 - Quick setup and installation on Ubuntu 18.04