Hoods are only inserted into the database if they are active,
inactive hoods are ignored.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
While we can have both br-client and br-mesh on top of the netif
list, we can only pre-load the data for one of them. This led to
the situation where br-mesh was shown as selected, but no data was
displayed.
Thus, let's just put the new br-client at that priviledged position,
and having br-mesh deselected by default. This should make the
situation more understandable.
Fixes: bd8d0280f6 ("treewide: add support for br-client")
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Add support for showing available memory if included in the alfred
dataset. This might be more helpful than free memory in several
cases.
While at it, tidy up and improve the JS code for the graph. It also
shows the latest values in the legend now.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
br-mesh has been renamed to br-client in recent firmware.
Implement br-client parallel to br-mesh.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Like done recently for the router details page, also reduce the
amount of plain data by using short names for the keys in json
stats. While at it, also remove unused fields and improve the
formatting of the query.
This fixes the traffic graph on the statistics page, which was
broken due to the earlier change for the routers.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
The commit was missing after the MySQL queries, leaving e.g. updated
V2 Hoods unchanged in the database.
Add commit for individual function and for the script to be sure
nothing else is overlooked.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Since part of the stats are just added to the html page, make
their keys shorter so the overall data transmitted per page is
reduced.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
init_influx.py: This should be run initially when the database is
set up.
update_influx.py: This should be run when the retention policies
are changed.
status_influx.pw: This will print the current retention policies
from the database.
For the scripts to work, the dictionary with retention policies
needs to be set up in influconfig.py, like shown in the example
file. Furthermore, the hostname, port, and database need to be
set correctly there, too (but not the user/pw, see below).
Since typically the user for the monitoring itself would not be
an admin, but admin access is required for changing the policies,
the scripts would be started by specifying the proper user name
as argument, e.g.
init_influx.py username
If auth_enabled = false, no user needs to be specified, and the
scripts may just be run without arguments.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
MySQL implementation of stats tables needed a dictionary of netifs.
This can be removed now.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Previously, data was sent to influxdb individually for each router.
This is expected to be inefficient, so let's collect all additions
per retention policy now (the latter is required by influxdb).
The downside is that the whole bunch will fail if one dataset is
broken. However, this only applies to the data after parsing the
XML/JSON. If parsing fails, this will be handled as before.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
So far, MySQL commits were triggered for each bunch of 500 devices.
However, since data is provided _per hood_, we will never receive
chunks that big. Just commit once after each alfred batch.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
This increases the chunk size for deletions and makes it possible
to change the value in one place.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
The previous implementation used an INNER JOIN for deletion.
This only works if there is an entry in all connected tables,
leaving a lot of devices undeleted after a certain timespan.
To really get all devices, this changes to just deleting from
the main table. The remnants in other tables can be dealt with
by another function to find orphaned entries later.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
So far, tiles have been stored on disk which, in combination with
bad disk performance, made the map slow and laggy.
This patch switches to memcache, which will remove the dependency
on disk performance for loading cached tiles entirely.
Note that installing and configuring memcached will be required
manually like for the other dependencies.
Signed-off-by: Fabian Bläse <fabian@blaese.de>
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
There should be no installations of the mongoDB-based Monitoring
left, so the upgrading instruction seem useless and might actually
be even misleading.
Remove them.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Based on a request, the URL /api/ssidclient will provide a plain-text,
new-line-separated list of all used AP SSIDs.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
We are using an ancient release of the leaflet framework.
Therefore it is updated to the latest compatible version.
Signed-off-by: Fabian Bläse <fabian@blaese.de>
[resolve carriage return rebase issue]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
We are using an ancient release of both the Bootstrap framework.
Therefore it is updated to the latest compatible version.
Signed-off-by: Fabian Bläse <fabian@blaese.de>
This adds the new old-stats tables to the script looking for lost
router-dependent entries.
While at it, add a short sleep between queries.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Since we changed the number of days stored for gateway statistics,
it makes sense to also have a (reduced) separate display config
variable.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Accumulated gateway stats are much less accurate and helpful than
those for hoods or the global ones. Since they however account for
most of the global stats data, reduce their history in patch.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
To reduce the amount of router stats, this introduces a new set
of router_stats_* tables that store data in 30-minutes-intervals.
The original tables remain, but the data in those will be kept for
a shorter time.
Thus, we have more dense data for recent stats and less dense data
for older stats. By reducing the density from every 5 to every 30
minutes, we reduce size of the "old" data to a sixth of the initial
amount. This should speed up I/O substantially.
Having two tables per category is superior to trimming the data in
one table: Remove data by e.g. modulo queries is slow and will
cause strong fragmentation of the table. By having two tables, we
introduce same data overlap, but on the other hand effectively
provide table partitioning making individual tables smaller.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
So far, response was defined in try block, but also used in except
block. This resulted in variable-not-set errors when exception was
raised before the response was set.
This sets response before try block to have it defined in any case.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
This logs cases where the netif is not found in table and thus
AUTO_INCREMENT in netifs table will be raised.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
This fixes errors like the following:
{2019-07-01 19:40:01} - Warning: Exception occurred when saving xxxx: division by zero
__Traceback (most recent call last):
__ File "/usr/local/lib/python3.5/dist-packages/ffmap/routertools.py", line 160, in import_nodewatcher_xml
__ calculate_network_io(mysql, router_id, uptime, router_update)
__ File "/usr/local/lib/python3.5/dist-packages/ffmap/routertools.py", line 669, in calculate_network_io
__ netif_update["traffic"]["rx"] = int(netif_update["traffic"]["rx_bytes"] / router_update["sys_uptime"])
__ZeroDivisionError: division by zero
__
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
This adds the babel version to the router database and displays
it in router.html.
While at it, have both babel version and BATMAN version appear only
if data is present.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Openstreemaps tile servers do not support IPv6.
They are also quite slow sometimes.
Therefore the default tile source is changed to RRZE servers
which do support IPv6 and are way faster.
Signed-off-by: Fabian Bläse <fabian@blaese.de>
Thunderforest Maps are proprietary and require an api key.
As I quite like the Wikimedia map style as it has a very low
contrast which makes nodes and connections between them stand out
way more.
URLs for the Thunderforest Map will point to Wikimedia now.
Signed-off-by: Fabian Bläse <fabian@blaese.de>
[Do not introduce new id to source URL parameter]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
This changes mechanism for legacy alfred to remove the 64 array
already in Monitoring and send to the same script as for alfred2.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
This introduces a new config variable global_stat_show_days that
will limit the displayed number of days on statistics.html.
The saved amount of data will be untouched.
This should speed up loading that page significantly.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
This sends alfred/alfred2 data to Christian's test scripts to
check for performance of a Grafana/InfluxDB setup.
This will be removed after testing.
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Adrian Schmutzler
Fabian Bläse