Commit Graph

520 Commits

Author SHA1 Message Date
Adrian Schmutzler d3c8a7a64d uwsgi: Suppress logging every request
Errors (Status 4xx/5xx) will still be logged.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-11-22 18:30:58 +01:00
Adrian Schmutzler 4bfe42bb67 map: Indicate WAN uplink with white center in dots
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-11-22 18:30:22 +01:00
Adrian Schmutzler 79bada38bb Changelog: Update
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:11:07 +02:00
Adrian Schmutzler d779b10778 config.py: Use dedicated folder for log files
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:10:16 +02:00
Adrian Schmutzler e46380eb50 config.py: Wait 15 minutes before offline status
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:10:16 +02:00
Adrian Schmutzler c7ebe21caa config.py: Adjust gw stats collection (only 1 day)
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:10:16 +02:00
Adrian Schmutzler 7a03e43c3b config.py: Adjust netif stats collection (14 days every 5 min)
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:10:16 +02:00
Adrian Schmutzler 339eaee9a5 scripts: Redirect cron output to log only
This prevents information sent via e-mail.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:10:15 +02:00
Adrian Schmutzler cf3517d9d2 api/alfred: Fix check for existing router and delete fragments
The router id "router_id" is only evaluated based on the
router_netif table. If the corresponding entry in the router
table is missing, an error occurs.

To deal with that, we now use the "olddata" variable for ifs,
which is initialized based on the router table. If nothing is
found there, we trigger delete_router to get rid of fragments
in other tables.

The latter is necessary, as we identify routers by MAC addresses
and thus old entries will keep to be a problem if just a new
entry is added to router_netif.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:10:15 +02:00
Adrian Schmutzler 6d492a3a25 api/alfred: Support Babel neighbor IP address inside tag
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:10:15 +02:00
Adrian Schmutzler c01a3017a2 api/alfred: Support loadavg5
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:10:15 +02:00
Dominik Heidler 22df21dc7e router.html: Improve readability of selected mesh network devs
Signed-off-by: Dominik Heidler <dominik@heidler.eu>
2018-07-03 14:10:15 +02:00
Adrian Schmutzler 323a3a000c api/alfred: Improve retrieval of L3 mac address from IPv6
Thanks to Fabian Blaese.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:10:15 +02:00
Adrian Schmutzler b1eacf5f0a maptools: Always use "best" connection for link color
This is much easier to maintain and prevents from having a
wrong average if e.g. w2mesh and w5mesh are present, but only
the better one is used.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:10:15 +02:00
Adrian Schmutzler c155dbef6e map/router.html: Show neighbor links in black
This improves contrast to the background

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 14:10:08 +02:00
Adrian Schmutzler 166cad518a map/router.html: Indicate ethernet mesh connections by color
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 13:49:41 +02:00
Adrian Schmutzler 3034fd9e2c maptools: Treat case of multiple neighbor interfaces correctly
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 13:46:30 +02:00
Adrian Schmutzler f6e9aec960 init_db: Add missing gw tables
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 13:45:40 +02:00
Adrian Schmutzler aa99035e23 statistics.html: Show aggregated traffic for hoods and total
This patch aims at showing the client-caused traffic. We use
bat0 for this, at this seems to be the easiest way which does not
require router-specific ifs etc.

This patch distinguishes between routers and gateways:

- For routers, we just use the bat0 data
- For gateways, we aggregate eth0.1, eth1.1, w2ap and w5ap

This requires changes to the MySQL database!

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 13:45:40 +02:00
Adrian Schmutzler b21b6c9201 router.html: Display Gateway
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 13:45:37 +02:00
Adrian Schmutzler 5e60c70ca5 api/alfred: Introduce gateway field
This introduced a boolean gateway field, which is set based on
the gateway connections sent via alfred.

If a device provides no gateways which it is connected to, it is
assumed to respresent a gateway.

This requires changes to the MySQL database!

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-07-03 13:45:29 +02:00
Adrian Schmutzler 092a94500b Changelog: Update
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:23:32 +02:00
Adrian Schmutzler eeac779589 router.html: Three columns for legend in neighbor plot
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:21:51 +02:00
Adrian Schmutzler b6b1b801fe router.html: Label historic neighbors in plot with "(old)"
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:21:42 +02:00
Adrian Schmutzler 6a85c59ce6 router.html: Display name for all neighbors (including "historic")
This change the behavior concerning the displayed netif name in
the legend of the neighbor stats plot. Previously, the netif name
of the device was shown, now we show the netif name corresponding
to the neighbor. This is necessary, as we do not log netif names
for neighbors.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:21:32 +02:00
Adrian Schmutzler 54764f7f43 router.html: Show former neighbors in neighbor stats plot
If a router is currently not connected as neighbor, we don't see
its history. This patch shows all current and former neighbors.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:18:22 +02:00
Adrian Schmutzler 7955e60f2f scripts: Add scripts to defragment MySQL tables
scripts/defragtables.py:
- If run without argument, all tables EXCEPT stats are defragmented
  (quick run)
- If run with argument e.g. "1", all table INCLUDING stats are
  defragmented (will take about one hour)

scripts/defragtable.py <space-separated list of tables>:
- Defragments the specified tables; will crash if table does not
  exist

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:12:47 +02:00
Adrian Schmutzler fc9a494078 routertools: Also delete from router_gw and router_stats_gw
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:12:39 +02:00
Adrian Schmutzler a20a473bb4 graph.js: Move reset button to the bottom left
This prevents overlay with large legends.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:12:10 +02:00
Adrian Schmutzler af2e2591b9 statistics.html: Show "models per client" pie chart
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:12:03 +02:00
Adrian Schmutzler 7203d594f3 map: Add extra layer for position popup
By default, position popup is off again until you select the layer:
With the layer enabled, behavior is as before.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:11:38 +02:00
Adrian Schmutzler 2fcd8af88e router.html: Only load netif data when clicked
The data for each netif is only loaded when the respective row
is clicked.
Correspondingly, br-mesh is loaded initially.

So far, it is unclear how big the impact of the netif filter in
MySQL transactions is, as those have no key of their own.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:10:53 +02:00
Adrian Schmutzler ab89b6a144 router.html: Don't load full neighbor stats, but only on demand
With this patch, only the neighbor stats for the last day are
loaded by default. If you want more, a hyperlink is implemented
for this purpose.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 13:07:20 +02:00
Adrian Schmutzler c7ee598e16 statictics.html: Show router models with smaller share
This also enables one decimal place for tooltips.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:50:36 +02:00
Adrian Schmutzler f2d9cc590d README: Add dependency and cron script call
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:50:27 +02:00
Adrian Schmutzler e2011c0808 config and router.html: Increased number of stored events
Since events do not cost much, the number stored and displayed
is drastically increased.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:50:11 +02:00
Adrian Schmutzler 8eb207e04e api/alfred: Suppress duplicate key errors for stats (workaround!)
Sometimes two queries want to insert the same entry into the
router stats table, although we check for that right before the
query is made. One can suppress this by using ON DUPLICATE KEY UPDATE
to ignore the second (redundant) entry.

This is no fix, but will work until this is examined more thoroughly.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:50:01 +02:00
Adrian Schmutzler 2b17115253 config: Reduce offline_threshold to 10 minutes
Since the gaps between alfred calls have been fixed, we can
reduce the waiting time before a router gets offline.

We now only tolerate a single missing data point, but not two
like before.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:49:53 +02:00
Adrian Schmutzler a57d459c77 api/alfred: Disable router_rate_limit_list
With router_rate_limit_list, routers were not processed if the
time difference between calls was less than 5 minutes for the
same MAC address.

While this is generally not bad, there are some drawbacks:
- Not having been aware of this fact, we have established other
  mechanisms to dilute data density, which might have interfered
- With KeyXchangeV2, two gateways will send data with less than
  5 min. difference. As gateways are not connected, we know that
  we alternately receive newer and older data. With
  router_rate_limit_list, some of this data has been discarded
  before its "age" was evaluated. This caused an unwanted additional
  dilution of data which might have caused "offline" routers not
  being actually offline (for a short period)
- With KeyXchangeV1, if the second call was a little earlier, the
  a big share of the data would not be "new enough" and just be
  discarded
- With KeyXchangeV1, the same would happen for the order of records
  varying between alfred calls, were some records would have more and
  some less than 5 minutes time difference

To get rid of these issues, we remove router_rate_limit_list and
test whether the newer measures to dilute data are effective.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:49:34 +02:00
Adrian Schmutzler c77e04ee7f api/alfred: Evaluate utcnow() only once for the whole alfred dataset
Since the time is used as key in MySQL, this might help to solve key
race conditions where new_router_stats is entered at the same time.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:49:23 +02:00
Adrian Schmutzler d1fd5e2bf9 api/alfred: Optimize delete queries for IPv6
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:48:28 +02:00
Adrian Schmutzler 68dff475d2 api/alfred: Use individual transactions to delete router data
This patch uses smaller (but more) transactions to delete router
data. This makes lock more specific and thus prevents deadlocks
quite effectively.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:45:44 +02:00
Adrian Schmutzler b93ed81c34 Enforce https for login page link
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:45:24 +02:00
Adrian Schmutzler f50fb4ed6e router.html: Add maximum value for airtime plot
This will cut defective 5 GHz data bigger than 100 %.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:45:15 +02:00
Adrian Schmutzler d6b7ce8e1e config: Introduce specific stats deletion threshold for gw history
Value set to only 7 days to reduce database size, since these
data seem to be of little relevance.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:45:01 +02:00
Adrian Schmutzler e499315dec api: Introduce dnslist and dnsentries
This provides lists of V2 routers to be used in DNS servers:

/api/dnslist - Plain tab-separated list to be used in custom scripts

/api/dnsentries - List of zone file entries without a header

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:44:29 +02:00
Adrian Schmutzler c8178beedd gwinfo and scripts/deletestats: Delete old netif data
Netif information is deleted 48 hours after the MAC addresses
have changed.

This requires changes to the MySQL database!

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-04-16 12:42:35 +02:00
Adrian Schmutzler 1fa8cf3206 Changelog: Update
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-02-22 16:53:21 +01:00
Adrian Schmutzler 8b97e05af5 router_list.html: Make uptime sortable
Offline routers are assumed to have uptime=0.

Minor design changes included.

Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-02-22 16:53:15 +01:00
Adrian Schmutzler b9aab43459 router_list.html: Show last contact
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2018-02-22 16:53:03 +01:00