Commit Graph

513 Commits

Author SHA1 Message Date
339eaee9a5 scripts: Redirect cron output to log only
This prevents information sent via e-mail.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:10:15 +02:00
cf3517d9d2 api/alfred: Fix check for existing router and delete fragments
The router id "router_id" is only evaluated based on the
router_netif table. If the corresponding entry in the router
table is missing, an error occurs.

To deal with that, we now use the "olddata" variable for ifs,
which is initialized based on the router table. If nothing is
found there, we trigger delete_router to get rid of fragments
in other tables.

The latter is necessary, as we identify routers by MAC addresses
and thus old entries will keep to be a problem if just a new
entry is added to router_netif.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:10:15 +02:00
6d492a3a25 api/alfred: Support Babel neighbor IP address inside tag
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:10:15 +02:00
c01a3017a2 api/alfred: Support loadavg5
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:10:15 +02:00
Dominik Heidler
22df21dc7e router.html: Improve readability of selected mesh network devs
Signed-off-by: Dominik Heidler <dominik@heidler.eu>
2018-07-03 14:10:15 +02:00
323a3a000c api/alfred: Improve retrieval of L3 mac address from IPv6
Thanks to Fabian Blaese.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:10:15 +02:00
b1eacf5f0a maptools: Always use "best" connection for link color
This is much easier to maintain and prevents from having a
wrong average if e.g. w2mesh and w5mesh are present, but only
the better one is used.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:10:15 +02:00
c155dbef6e map/router.html: Show neighbor links in black
This improves contrast to the background

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:10:08 +02:00
166cad518a map/router.html: Indicate ethernet mesh connections by color
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 13:49:41 +02:00
3034fd9e2c maptools: Treat case of multiple neighbor interfaces correctly
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 13:46:30 +02:00
f6e9aec960 init_db: Add missing gw tables
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 13:45:40 +02:00
aa99035e23 statistics.html: Show aggregated traffic for hoods and total
This patch aims at showing the client-caused traffic. We use
bat0 for this, at this seems to be the easiest way which does not
require router-specific ifs etc.

This patch distinguishes between routers and gateways:

- For routers, we just use the bat0 data
- For gateways, we aggregate eth0.1, eth1.1, w2ap and w5ap

This requires changes to the MySQL database!

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 13:45:40 +02:00
b21b6c9201 router.html: Display Gateway
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 13:45:37 +02:00
5e60c70ca5 api/alfred: Introduce gateway field
This introduced a boolean gateway field, which is set based on
the gateway connections sent via alfred.

If a device provides no gateways which it is connected to, it is
assumed to respresent a gateway.

This requires changes to the MySQL database!

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 13:45:29 +02:00
092a94500b Changelog: Update
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:23:32 +02:00
eeac779589 router.html: Three columns for legend in neighbor plot
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:21:51 +02:00
b6b1b801fe router.html: Label historic neighbors in plot with "(old)"
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:21:42 +02:00
6a85c59ce6 router.html: Display name for all neighbors (including "historic")
This change the behavior concerning the displayed netif name in
the legend of the neighbor stats plot. Previously, the netif name
of the device was shown, now we show the netif name corresponding
to the neighbor. This is necessary, as we do not log netif names
for neighbors.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:21:32 +02:00
54764f7f43 router.html: Show former neighbors in neighbor stats plot
If a router is currently not connected as neighbor, we don't see
its history. This patch shows all current and former neighbors.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:18:22 +02:00
7955e60f2f scripts: Add scripts to defragment MySQL tables
scripts/defragtables.py:
- If run without argument, all tables EXCEPT stats are defragmented
  (quick run)
- If run with argument e.g. "1", all table INCLUDING stats are
  defragmented (will take about one hour)

scripts/defragtable.py <space-separated list of tables>:
- Defragments the specified tables; will crash if table does not
  exist

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:12:47 +02:00
fc9a494078 routertools: Also delete from router_gw and router_stats_gw
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:12:39 +02:00
a20a473bb4 graph.js: Move reset button to the bottom left
This prevents overlay with large legends.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:12:10 +02:00
af2e2591b9 statistics.html: Show "models per client" pie chart
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:12:03 +02:00
7203d594f3 map: Add extra layer for position popup
By default, position popup is off again until you select the layer:
With the layer enabled, behavior is as before.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:11:38 +02:00
2fcd8af88e router.html: Only load netif data when clicked
The data for each netif is only loaded when the respective row
is clicked.
Correspondingly, br-mesh is loaded initially.

So far, it is unclear how big the impact of the netif filter in
MySQL transactions is, as those have no key of their own.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:10:53 +02:00
ab89b6a144 router.html: Don't load full neighbor stats, but only on demand
With this patch, only the neighbor stats for the last day are
loaded by default. If you want more, a hyperlink is implemented
for this purpose.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:07:20 +02:00
c7ee598e16 statictics.html: Show router models with smaller share
This also enables one decimal place for tooltips.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:50:36 +02:00
f2d9cc590d README: Add dependency and cron script call
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:50:27 +02:00
e2011c0808 config and router.html: Increased number of stored events
Since events do not cost much, the number stored and displayed
is drastically increased.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:50:11 +02:00
8eb207e04e api/alfred: Suppress duplicate key errors for stats (workaround!)
Sometimes two queries want to insert the same entry into the
router stats table, although we check for that right before the
query is made. One can suppress this by using ON DUPLICATE KEY UPDATE
to ignore the second (redundant) entry.

This is no fix, but will work until this is examined more thoroughly.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:50:01 +02:00
2b17115253 config: Reduce offline_threshold to 10 minutes
Since the gaps between alfred calls have been fixed, we can
reduce the waiting time before a router gets offline.

We now only tolerate a single missing data point, but not two
like before.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:49:53 +02:00
a57d459c77 api/alfred: Disable router_rate_limit_list
With router_rate_limit_list, routers were not processed if the
time difference between calls was less than 5 minutes for the
same MAC address.

While this is generally not bad, there are some drawbacks:
- Not having been aware of this fact, we have established other
  mechanisms to dilute data density, which might have interfered
- With KeyXchangeV2, two gateways will send data with less than
  5 min. difference. As gateways are not connected, we know that
  we alternately receive newer and older data. With
  router_rate_limit_list, some of this data has been discarded
  before its "age" was evaluated. This caused an unwanted additional
  dilution of data which might have caused "offline" routers not
  being actually offline (for a short period)
- With KeyXchangeV1, if the second call was a little earlier, the
  a big share of the data would not be "new enough" and just be
  discarded
- With KeyXchangeV1, the same would happen for the order of records
  varying between alfred calls, were some records would have more and
  some less than 5 minutes time difference

To get rid of these issues, we remove router_rate_limit_list and
test whether the newer measures to dilute data are effective.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:49:34 +02:00
c77e04ee7f api/alfred: Evaluate utcnow() only once for the whole alfred dataset
Since the time is used as key in MySQL, this might help to solve key
race conditions where new_router_stats is entered at the same time.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:49:23 +02:00
d1fd5e2bf9 api/alfred: Optimize delete queries for IPv6
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:48:28 +02:00
68dff475d2 api/alfred: Use individual transactions to delete router data
This patch uses smaller (but more) transactions to delete router
data. This makes lock more specific and thus prevents deadlocks
quite effectively.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:45:44 +02:00
b93ed81c34 Enforce https for login page link
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:45:24 +02:00
f50fb4ed6e router.html: Add maximum value for airtime plot
This will cut defective 5 GHz data bigger than 100 %.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:45:15 +02:00
d6b7ce8e1e config: Introduce specific stats deletion threshold for gw history
Value set to only 7 days to reduce database size, since these
data seem to be of little relevance.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:45:01 +02:00
e499315dec api: Introduce dnslist and dnsentries
This provides lists of V2 routers to be used in DNS servers:

/api/dnslist - Plain tab-separated list to be used in custom scripts

/api/dnsentries - List of zone file entries without a header

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:44:29 +02:00
c8178beedd gwinfo and scripts/deletestats: Delete old netif data
Netif information is deleted 48 hours after the MAC addresses
have changed.

This requires changes to the MySQL database!

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:42:35 +02:00
1fa8cf3206 Changelog: Update
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-02-22 16:53:21 +01:00
8b97e05af5 router_list.html: Make uptime sortable
Offline routers are assumed to have uptime=0.

Minor design changes included.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-02-22 16:53:15 +01:00
b9aab43459 router_list.html: Show last contact
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-02-22 16:53:03 +01:00
c581915076 api/alfred: Remove router hardname name correction for old FW
This only affects 0.5.1 and older.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-02-22 16:46:52 +01:00
488a8a2c63 router.html: Show warning if no contact address is set
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-02-22 16:46:41 +01:00
5d7e00422e api/alfred: Check for new router data initially by system time
If we receive data from more than one gateway, there happens to
be a mix of older and newer data (since synchronization between
gateways seems to be not working).

To deal with that, we now only accept data where the router's
system time is newer than the value stored in the DB. To account
for time synchronization issues, we also accept data which is more
than one hour older.

This patch removes other checks for old data which are now obsolete.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-02-22 16:46:31 +01:00
d3ea76b648 scripts/deletestats: Use block-wise delete for more stats tables
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-02-22 16:46:20 +01:00
c208663f70 router.html/map: Treat quality differently based on routing protocol
Adds display support for BATMAN_V data.

This is step 1 of 2. It does change the background colors for
neighbors, but does NOT change the link colors in the map.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-02-22 16:45:40 +01:00
f6f699c8c7 api/alfred and router.html: Include routing_protocol
This only works for routers with updated nodewatcher.

This requires changes to the MySQL database!

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-02-22 16:45:32 +01:00
d82e7bf5a7 map: Only show coordinates on second click
First click closes open popup.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-02-22 16:45:22 +01:00