Commit Graph

86 Commits

Author SHA1 Message Date
Adrian Schmutzler 55dc7b4ad5 Rename net_if from router_neighbor table to netif
This includes various changes throughout the code. Additionally,
some fields are reordered.

This requires a change of the MySQL table router_neighbor!

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-30 18:21:08 +01:00
Adrian Schmutzler 39d9ddc9d0 api/alfred: Update router data instead of DELETE/INSERT
Previously, I just deleted all entries and recreated them again
with INSERT. Although this is simple to write and actually includes
less queries, it causes a lot more write IO. Since most of the
neighbors and interfaces do NOT change frequently, it is worth the
extra effort to delete only those really gone since the last
update.

Only br-mesh will normally have assigned IPv6 addresses, thus
we just delete all IPv6 adresses of the other ones.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-30 18:21:08 +01:00
Adrian Schmutzler 55f81c4295 api/alfred: Change calculation of rx/tx after reboot
Previously, on restart the traffic of the last period before
the restart was reused.

Now, we use the logged traffic divided by the uptime.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-30 18:19:24 +01:00
Adrian Schmutzler 858f419e54 api/alfred and router.html: Show WiFi data for netifs
This shows information about WiFi parameters (e.g. channel).

Except the Tx-Power, data is only available if a firmware with
a corresponding nodewatcher update is present (version 44).

This requires a change of the MySQL table router_netif!

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-30 18:08:41 +01:00
Adrian Schmutzler 7ef6f47c4e routertools: Reorder fields in INSERT queries for router stats
This is a cosmetic change to realign the INSERTs to the database.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-30 18:04:12 +01:00
Adrian Schmutzler 58ce32e322 Change router_stats_netif to use ids for netifs
This introduces a serious of changes to code and database.

This patch requires changes to the MySQL database.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-30 17:18:22 +01:00
Adrian Schmutzler a3ee0edead routertools/alfred: Replace empty hostname
Fixes #109

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-21 17:06:59 +01:00
Adrian Schmutzler 20e71afeb0 Provide possibility to ban routers
If routers are supposed to be removed from the Monitoring
permanently, they can now be banned based on their MAC address.

All admins can do that via the web interface.

ATTENTION: This requires a database update!

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-21 17:03:27 +01:00
Adrian Schmutzler 636a8d3baa routertools: Use custom file for full log, do not write to syslog
It is easier to evaluate a specific log which is not crowded with
messages from other programs

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-10 20:25:16 +01:00
Adrian Schmutzler 1c80bafc10 routertools/alfred: New routers are always 'online'
Since other states are only evaluated correctly on a CHANGE, we
set status to 'online' for new routers, so that the second call
can log unknown events.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-10 20:25:16 +01:00
Adrian Schmutzler 69fbcd3181 routertools/alfred: Catch malformatted coordinates
The puts a router with malformatted coordinates into the default
hood.

Note:
For a comma instead of dot (lat=48,3), the Monitoring will assign
default, while the keyserver will use floor(), resulting in
lat=48.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-10 20:25:16 +01:00
Adrian Schmutzler 53c9dc11fd routertools/alfred: Improve exception logging
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-10 20:25:16 +01:00
Adrian Schmutzler 7a678a73c6 routertools/alfred: Set status earlier
This prevent a variable-not-found error when status is not set
in except statement.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-10 20:25:16 +01:00
Adrian Schmutzler 48bf9c008b calcglobalstats/deletestats: Improve duration logging and output
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-10 20:25:00 +01:00
Adrian Schmutzler 80f20f4d0c routertools/deletestats: Split and improve netif stats deletion
This patch reorganizes the deletion of old stats:
- Commits are done after each step
- Netif stats deletion is split into UPDATE and DELETE
- Delays are added, to reduce locking

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-10 20:04:10 +01:00
Adrian Schmutzler f52af65c1c routertools/config: Add minimum time difference for router stats
This prevents errors due to the same router being sent twice in
the same second.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-10 19:52:02 +01:00
Adrian Schmutzler 9e6e3c8dbf routertools/alfred: Minimize time without router_netif data
Rearrange queries to have a minimum time where netifs for router
are not present.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-10 19:51:49 +01:00
Adrian Schmutzler a994ec114a MySQL stats: Convert from datetime to int
DB scripts still need to be updated.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-10 19:49:46 +01:00
Adrian Schmutzler a1024baea0 routertools/alfred: Recognize MySQL lock as specific exception
This will not change the router status to unknown

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-12-10 19:44:33 +01:00
Adrian Schmutzler d0d173d935 Introduce debugging function
This adds a timestamp to all debug outputs

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-23 22:02:27 +01:00
Adrian Schmutzler 846f66281e routertools: Don't delete old stats of offline routers
With this patch, router stats are only deleted if the router
is online or missing in the router table.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-23 22:01:44 +01:00
Adrian Schmutzler 42c78f1079 routertools: Remove commented old code
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-23 22:01:29 +01:00
Adrian Schmutzler 8f04e13c24 routertools/router.html: Display warning for lost coordinates
Warning: This requires a change of the MySQL table 'router'

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-23 21:59:30 +01:00
Adrian Schmutzler ebb4589301 routertools: Remember settings after router reset
This restores the behavior before the MySQL conversion: If a
router is reset, the new (empty) values are not written to
the variables, instead the old ones are kept.

In contrast to the old setting, however, we still do reset the
hood.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-23 21:56:35 +01:00
Adrian Schmutzler 7f81bff24b routertools: Don't use old coordinates for hood assignment
If no lat/lng or hood is sent, the device should go to Default.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-23 21:56:18 +01:00
Adrian Schmutzler 07208b6f5c routertools: Reorganize parsed data from router XML
Since we do not use MongoDB anymore, the structure of the parsed
data in router_update can be simplified to a less nested scheme.

In addition to that, directory keys have been adjusted to those
in MySQL. This prevents typos and errors.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-23 21:48:48 +01:00
Adrian Schmutzler e20d37edf9 routertools: Write exceptions for parsing routers to file
Since exceptions during parsing a router's alfred data are
caught, you do not learn about the reasons.

Thus, this writes some information to a file.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-20 11:48:55 +01:00
Adrian Schmutzler 8ff508f211 routertools: Remove leftover from debugging
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-20 11:48:49 +01:00
Adrian Schmutzler 5e75f9cad0 config: Move config to a single, separate file
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-19 15:30:39 +01:00
Adrian Schmutzler 595810150c routertools: Reduce offline delay to 15 minutes
This is possible due to the optimized alfred proxy cron
triggers.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-19 15:26:35 +01:00
Adrian Schmutzler 3fd731a5a2 Introduced orphaned state between offline and deletion
After 7 days of being offline, a router enters the orphaned
state with a grey icon. It is only deleted after a longer period
of 180 days.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-16 20:41:38 +01:00
Adrian Schmutzler 39bf2032f4 routertools: Aggregate INSERT commands using executemany
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-16 20:37:26 +01:00
Adrian Schmutzler ece82c44f4 routertools: Remove direct references to mysql cursor
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-16 14:50:16 +01:00
Adrian Schmutzler 003fcbcebe routertools/stattools: Implement default hood as str "Default"
If the default is NULL (as previously), we have ugly problems
with indexing and queries. To circumvent this, the hood is
set to "Default" right at the beginning.

For old data, we add an if to the hood stats calculation.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-16 00:05:48 +01:00
Adrian Schmutzler 48cb9f0033 routertools: Remove lower() for hood name
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-16 00:04:47 +01:00
Adrian Schmutzler 92cd0e00a8 routertools: Treat missing XML elements better
This particularly fixes the case of missing coordinates.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-16 00:02:03 +01:00
Adrian Schmutzler 9a32c7bffd Delete old router events
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-16 00:01:26 +01:00
Adrian Schmutzler e3fe995407 MySQL: alpha3
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-11-16 00:00:07 +01:00
Dominik Heidler 058bed9a2e Implement loadavg graph data collection for #55 2017-09-11 16:31:57 +02:00
Dominik Heidler f8c5913ad1 Commit unstaged changes from production instance 2017-08-21 11:49:26 +02:00
Adrian Schmutzler 0ac862411a Set WAN uplink based on new nodewatcher xml entry and for L2TP
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2017-06-17 21:48:08 +02:00
Steffen Pankratz 3318794d0f Added support for layer 3 links (fixes #38)
Signed-off-by: Steffen Pankratz <kratz00@gmx.de>
2017-04-17 13:31:31 +02:00
Steffen Pankratz ff4b8ebd8f - deleted functionality to fetch data from Netmon
Signed-off-by: Steffen Pankratz <kratz00@gmx.de>
2017-03-22 18:14:54 +01:00
Dominik Heidler b16199c8db Use Hood from Router XML if provided
Fixes #31
2017-03-21 10:06:59 +01:00
Steffen Pankratz 4b09731360
Added general exception handler for function 'import_nodewatcher_xml'
Signed-off-by: Steffen Pankratz <kratz00@gmx.de>
2017-01-28 18:20:36 +01:00
Dominik Heidler d940526f79 Get chipset value only if XML key exists 2016-07-26 22:20:59 +02:00
Dominik Heidler da42ef018c Add in memory router rate limit
This is some kind of tmp hack to reduce db load
2016-07-26 22:17:39 +02:00
Dominik Heidler 797b67ee40 Record smaller stat timespan to reduce db load 2016-07-26 22:16:33 +02:00
Dominik Heidler 3bf3445c51 Merge pull request #18 from kratz00/empty_mac
Added workaround for empty 'mac_addr' element in nodewatcher data.
2016-05-16 17:03:24 +02:00
Steffen Pankratz 45943a5246 Added workaround for empty 'chipset' element in nodewatcher data.
Signed-off-by: Steffen Pankratz <kratz00@gmx.de>
2016-05-16 12:20:03 +02:00