ejabberd - Monday, November 4, 2024

[16:13:41] <zeank> @Holger: you were right, it was the watchdog issue that causes my server to restart (crash) every now and then
[16:14:22] <Holger> zeank: Was that on Debian?

[16:14:25] kaptan joins the room

[16:15:10] <zeank> ish, it’s ubuntu

[16:15:56] Frank joins the room

[16:15:57] Frank leaves the room

[16:16:07] deimosBSD joins the room

[16:16:25] Squeaky Latex Folf leaves the room

[16:16:39] x00010 joins the room

[16:16:43] <Holger> zeank: If you have the chance/time, I'd be very interested if it looks like an AppArmor issue like someone suggested in the issue …

[16:16:43] ukko joins the room

[16:17:41] <zeank> I know nothing about AppArmor, how to investigate this?

[16:17:52] danielepiccoli leaves the room

[16:19:23] random_tester_123 leaves the room

[16:19:24] random_tester_123 joins the room

[16:19:44] stp joins the room

[16:20:07] neox joins the room

[16:20:11] neox leaves the room

[16:20:21] Licaon_Kter leaves the room

[16:20:52] insanity joins the room

[16:21:05] insanity leaves the room

[16:21:19] <Holger> zeank: First of all `sudo aa-status` should tell you whether it's enabled.

[16:22:16] insanity joins the room

[16:22:20] rain leaves the room

[16:22:21] rain joins the room

[16:22:41] neox joins the room

[16:22:44] insanity leaves the room

[16:23:09] <zeank> looks much like it

[16:23:45] insanity joins the room

[16:24:05] insanity leaves the room

[16:25:04] inkey joins the room

[16:25:12] insanity joins the room

[16:25:12] Jeroen joins the room

[16:25:14] Jeroen leaves the room

[16:25:19] inkey leaves the room

[16:25:26] insanity leaves the room

[16:26:04] <Holger> As per <https://wiki.debian.org/AppArmor/HowToUse#Diagnose_if_a_bug_might_have_been_caused_by_AppArmor> the following command should show whether AppArmor blocked some operation (assuming the Journal logs weren't expired since ejabberd was killed):
```
sudo journalctl | grep DENIED
```

[16:26:21] Frank joins the room

[16:26:22] Frank leaves the room

[16:26:27] <Holger> (Can't easily test things myself as I have AppArmor disabled.)

[16:26:28] Frank joins the room

[16:26:28] Frank leaves the room

[16:26:51] j.r leaves the room

[16:27:05] <zeank> No worries, here to learn :D

[16:27:11] ukko leaves the room

[16:27:50] <zeank> takes a while since it’s a lot of logs to go through I’m afraid
[16:28:09] <zeank> ejabberd is running on ‘info’ …
[16:28:22] <Holger> zeank, apart from that, if you're okay with that you could disable AppArmor temporarily (just for testing) while the systemd Watchdog is enabled and see whether the issue pops up nevertheless:
https://wiki.debian.org/AppArmor/HowToUse#Disable_AppArmor

[16:28:41] Jeroen joins the room

[16:28:41] Jeroen leaves the room

[16:28:49] <zeank> ok

[16:29:52] schäfchen726 leaves the room

[16:29:54] insanity joins the room

[16:30:19] insanity leaves the room

[16:30:24] <Holger> As I said in my comment on the issue, I'd think AppArmor is all-or-nothing so I don't quite get how AppArmor would break the Watchdog thing only occasionally. But it would be nice to rule it out.

[16:30:31] ukko joins the room

[16:30:36] <zeank> yeah

[16:31:23] insanity joins the room

[16:31:33] carlosr joins the room

[16:31:47] insanity leaves the room

[16:32:37] tobiasd leaves the room

[16:32:46] <zeank> found those here only and they are not around the time when it happened (also it’s docker, not using docker for ejabberd):
> Nov 04 08:15:18 calimero kernel: audit: type=1400 audit(1730708118.185:135): apparmor="DENIED" operation="signal" class="signal" profile="docker-default" pid=29704 comm="runc" requested_mask="receive" denied_mask="receive" signal=kill peer="runc"

[16:32:46] insanity joins the room

[16:33:16] <zeank> I’ll re-enable Watchdog and disable AppArmor, then we’ll see
[16:33:30] <Holger> Thanks a lot.

[16:33:39] leke leaves the room

[16:33:45] <zeank> no prob

[16:33:50] abba leaves the room

[16:34:39] leke joins the room

[16:35:11] insanity joins the room

[16:35:22] insanity leaves the room

[16:35:36] <Holger> The systemd code is mine so I feel bad about people running into that foo. Didn't manage to track it down so far. Wasn't able to reproduce myself. What I've seen from others is that the Erlang timer that triggers the Watchdog ping _does_ fire just fine, so it seems like there's something going wrong with the actual ejabberd→systemd ping (via Unix domain socket).

[16:35:36] zeank leaves the room

[16:35:57] <Holger> Occasionally.

[16:36:29] Frank joins the room

[16:36:30] Frank leaves the room

[16:36:31] insanity joins the room

[16:36:56] insanity leaves the room

[16:37:29] zeank joins the room

[16:38:02] <zeank> > root@calimero:~~# apparmor_status
apparmor module is loaded.
apparmor filesystem is not mounted.
[16:38:13] <zeank> how often do you ping?

[16:38:19] insanity joins the room

[16:39:14] tobiasd joins the room

[16:39:20] rozzin leaves the room

[16:39:35] <Holger> That's configured in the systemd unit (`WatchdogSec`), I think the default is 30 seconds. systemd hands the value over to ejabberd via environment variable, I think I use WatchdogSec/2 as interval.

[16:39:36] debalance joins the room

[16:40:06] uira leaves the room

[16:41:13] <Holger> https://github.com/processone/ejabberd/blob/601fcba4cbf881942fae4ab89bc8f666f7a9213f/src/ejabberd_systemd.erl#L152-L154

[16:41:36] mirux joins the room

[16:42:26] leke joins the room

[16:42:27] OOS-Bob joins the room

[16:42:41] tobiasd joins the room

[16:44:49] <zeank> ffiw I have
> /opt/ejabberd/logs/ejabberd.log:2024-11-04 16:36:17.816355+00:00 [info] <0.560.0>@ejabberd_systemd:maybe_start_timer/1:164 Watchdog notifications enabled

[16:45:29] Squeaky Latex Folf joins the room

[16:47:48] ukko leaves the room

[16:48:01] <zeank> not much to a surprise
> case os:getenv("WATCHDOG_USEC") of
                         WatchdogUSec when is_list(WatchdogUSec), length(WatchdogUSec) > 0 ->
                             Interval = round(0.5 * list_to_integer(WatchdogUSec)),
                             erlang:convert_time_unit(Interval, microsecond, millisecond);
                         _ ->
                             undefined
                           end.
15000

[16:48:30] ukko joins the room

[16:48:57] <Holger> Yup, looks correct.

[16:49:50] Sally joins the room

[16:49:56] <zeank> Also, there’s no load or anything else that looks suspicious on that machine

[16:51:04] ukko leaves the room

[16:51:45] Sally leaves the room

[16:53:24] carlosr leaves the room

[16:54:27] lauranna leaves the room

[16:54:31] lauranna joins the room

[16:54:39] zeank joins the room

[16:55:05] christian joins the room

[16:55:06] ukko joins the room

[16:56:07] carlosr joins the room

[16:56:40] bio leaves the room

[16:59:34] danielepiccoli joins the room

[16:59:47] lauranna leaves the room

[17:00:53] lauranna leaves the room

[17:01:38] Kris leaves the room

[17:02:56] leke leaves the room

[17:02:56] Tim leaves the room

[17:03:49] Kris joins the room

[17:03:58] Frank joins the room

[17:03:59] Frank leaves the room

[17:06:04] rain leaves the room

[17:06:28] leke leaves the room

[17:06:34] Tim joins the room

[17:06:45] ukko joins the room

[17:07:31] ij joins the room

[17:08:41] Titi leaves the room

[17:08:45] Titi joins the room

[17:09:37] leke joins the room

[17:10:58] brian leaves the room

[17:10:58] brian joins the room

[17:11:30] Holger leaves the room

[17:12:56] rain joins the room

[17:15:35] j.r joins the room

[17:15:38] leke leaves the room

[17:16:43] Titi leaves the room

[17:16:50] neox leaves the room

[17:16:57] neox joins the room

[17:17:00] targ leaves the room

[17:17:06] Titi joins the room

[17:17:16] leke joins the room

[17:19:09] zeank joins the room

[17:19:30] rain leaves the room

[17:22:07] leke joins the room

[17:22:21] nutcracker500 joins the room

[17:22:53] debalance leaves the room

[17:24:23] kaptan leaves the room

[17:26:48] EmleyMoor leaves the room

[17:27:25] rain joins the room

[17:27:55] EmleyMoor joins the room

[17:28:07] ukko leaves the room

[17:29:26] kaptan joins the room

[17:30:24] Kris leaves the room

[17:31:06] pod leaves the room

[17:32:48] ff leaves the room

[17:34:11] rain leaves the room

[17:35:56] cel leaves the room

[17:37:18] leke leaves the room

[17:37:49] Kris joins the room

[17:39:34] barlas leaves the room

[17:39:42] Citlali joins the room

[17:39:43] Sally joins the room

[17:40:58] Sally leaves the room

[17:42:05] stp leaves the room

[17:42:28] rain joins the room

[17:45:22] Titi leaves the room

[17:45:36] Titi joins the room

[17:47:26] leke leaves the room

[17:48:22] Ramiro Romani joins the room

[17:49:04] rain leaves the room

[17:49:39] Citlali leaves the room

[17:51:18] pitchum leaves the room

[17:51:41] Jeroen joins the room

[17:51:43] Jeroen leaves the room

[17:52:29] gooya joins the room

[17:54:30] leke joins the room

[17:56:06] nutcracker500 leaves the room

[17:56:34] mightyBroccoli joins the room

[17:58:21] Holger joins the room

[18:00:24] Holger leaves the room

[18:02:22] nutcracker500 joins the room

[18:03:12] rain joins the room

[18:03:30] Tim leaves the room

[18:04:23] ukko joins the room

[18:04:30] pitchum leaves the room

[18:07:14] bio joins the room

[18:08:22] polymath leaves the room

[18:09:47] rain leaves the room

[18:09:52] Licaon_Kter leaves the room

[18:09:56] Licaon_Kter joins the room

[18:10:05] mightyBroccoli leaves the room

[18:10:58] ukko leaves the room

[18:11:14] karme leaves the room

[18:13:04] allieali leaves the room

[18:13:05] allieali joins the room

[18:14:20] ukko joins the room

[18:15:02] VC leaves the room

[18:15:08] VC joins the room

[18:17:05] leke joins the room

[18:17:26] leke leaves the room

[18:17:29] erik joins the room

[18:20:24] Squeaky Latex Folf leaves the room

[18:21:07] egnun leaves the room

[18:21:09] egnun joins the room

[18:24:24] nutcracker500 leaves the room

[18:24:35] rain joins the room

[18:25:19] leke joins the room

[18:26:18] pitchum joins the room

[18:26:45] leke leaves the room

[18:28:35] pitchum joins the room

[18:29:00] debacle leaves the room

[18:30:06] mirux leaves the room

[18:33:30] Ramiro Romani leaves the room

[18:34:38] rain leaves the room

[18:34:48] Ramiro Romani leaves the room

[18:35:10] bio leaves the room

[18:35:49] Tim joins the room

[18:36:10] rain joins the room

[18:37:54] cel joins the room

[18:38:14] ff joins the room

[18:39:17] JonNJ leaves the room

[18:39:17] JonNJ joins the room

[18:41:58] Tim leaves the room

[18:42:28] rain leaves the room

[18:42:46] rain joins the room

[18:43:13] pitchum leaves the room

[18:43:48] kapad joins the room

[18:44:19] Jeroen joins the room

[18:44:20] Jeroen leaves the room

[18:46:10] Amolith joins the room

[18:46:17] Mario leaves the room

[18:47:02] Mario joins the room

[18:48:07] marc36 joins the room

[18:49:11] rain leaves the room

[18:49:54] Citlali joins the room

[18:51:20] allieali leaves the room

[18:52:17] leke leaves the room

[18:55:19] leke joins the room

[18:58:14] rain joins the room

[18:59:05] rintinfinn leaves the room

[18:59:59] lauranna joins the room

[19:00:07] lauranna joins the room

[19:01:37] 窃火者 leaves the room

[19:03:05] 窃火者 joins the room

[19:03:27] p42ity leaves the room

[19:04:20] rain leaves the room

[19:04:55] OOS-Bob leaves the room

[19:05:31] Wojtek leaves the room

[19:06:34] karme joins the room

[19:07:27] rain joins the room

[19:08:01] polymath joins the room

[19:08:12] Ramiro Romani joins the room

[19:08:26] bio joins the room

[19:08:42] lucas leaves the room

[19:09:10] Wojtek joins the room

[19:09:11] Wojtek leaves the room

[19:09:13] alacer joins the room

[19:09:16] Kris leaves the room

[19:09:32] Wojtek joins the room

[19:09:34] Wojtek leaves the room

[19:09:46] lucas joins the room

[19:09:55] Wojtek joins the room

[19:09:56] Wojtek leaves the room

[19:09:58] lucas joins the room

[19:10:04] lucas joins the room

[19:10:15] Núbio leaves the room

[19:10:16] Wojtek joins the room

[19:10:17] Wojtek leaves the room

[19:10:38] Wojtek joins the room

[19:10:39] Wojtek leaves the room

[19:10:58] alacer leaves the room

[19:13:07] lauranna leaves the room

[19:13:27] Núbio joins the room

[19:13:27] ukko leaves the room

[19:14:21] robert_mobil joins the room

[19:14:48] Kris joins the room

[19:14:52] serafean joins the room

[19:15:16] serafean leaves the room

[19:18:00] VC leaves the room

[19:18:02] VC joins the room

[19:18:49] serafean joins the room

[19:19:24] serafean leaves the room

[19:20:20] robert_mobil leaves the room

[19:20:24] p42ity joins the room

[19:20:26] p42ity leaves the room

[19:20:28] Ramiro Romani leaves the room

[19:20:39] p42ity joins the room

[19:21:30] leke leaves the room

[19:21:42] serafean joins the room

[19:22:05] serafean leaves the room

[19:23:04] Ramiro Romani joins the room

[19:24:51] bio leaves the room

[19:25:02] rain leaves the room

[19:25:27] robert_mobil joins the room

[19:26:12] rain joins the room

[19:26:35] Amolith leaves the room

[19:26:41] Amolith joins the room

[19:28:51] EmleyMoor leaves the room

[19:30:35] robert_mobil leaves the room

[19:31:01] ij leaves the room

[19:31:38] robert_mobil leaves the room

[19:31:45] EmleyMoor joins the room

[19:33:42] Wojtek joins the room

[19:33:43] Wojtek leaves the room

[19:34:02] Wojtek joins the room

[19:34:10] rintinfinn joins the room

[19:34:39] pitchum joins the room

[19:37:09] rozzin joins the room

[19:37:55] ij joins the room

[19:39:26] ukko leaves the room

[19:39:36] ukko joins the room

[19:46:03] Jeroen joins the room

[19:46:04] Jeroen leaves the room

[19:46:30] lauranna leaves the room

[19:50:25] cel leaves the room

[19:51:08] rom1dep joins the room

[19:51:33] rain leaves the room

[19:52:53] carlo leaves the room

[19:52:54] carlo joins the room

[19:52:59] alacer joins the room

[19:53:03] alacer leaves the room

[19:53:14] lauranna joins the room

[19:53:29] 窃火者 leaves the room

[19:53:35] serafean joins the room

[19:53:42] gooya leaves the room

[19:53:48] gooya joins the room

[19:54:37] alacer joins the room

[19:54:48] 窃火者 joins the room

[19:54:58] ukko joins the room

[19:55:04] serafean leaves the room

[19:56:42] alacer leaves the room

[19:57:03] john leaves the room

[19:57:10] john joins the room

[19:57:46] serafean joins the room

[19:58:30] serafean leaves the room

[19:58:44] Jeroen joins the room

[19:58:45] Jeroen leaves the room

[19:58:52] akshay leaves the room

[19:59:52] serafean joins the room

[20:00:34] serafean leaves the room

[20:00:50] rain joins the room

[20:02:13] mohamdds leaves the room

[20:02:50] Sally joins the room

[20:03:37] Wojtek leaves the room

[20:04:36] Jeroen joins the room

[20:04:37] Jeroen leaves the room

[20:05:02] Titi joins the room

[20:05:57] savagepeanut joins the room

[20:07:07] sarvo joins the room

[20:07:29] Mitex Leo leaves the room

[20:07:36] ermete joins the room

[20:10:55] abba joins the room

[20:12:15] Wojtek joins the room

[20:12:38] ermete leaves the room

[20:12:38] ermete joins the room

[20:14:57] Wojtek leaves the room

[20:16:52] fabio leaves the room

[20:17:12] abba leaves the room

[20:18:41] Marc leaves the room

[20:19:30] writer77 leaves the room

[20:20:19] isotelo leaves the room

[20:20:59] Marc joins the room

[20:21:50] marc leaves the room

[20:21:52] marc joins the room

[20:22:04] writer77 joins the room

[20:22:19] writer77 leaves the room

[20:23:02] marzzzello joins the room

[20:23:19] ukko leaves the room

[20:23:32] writer77 joins the room

[20:24:30] fabio joins the room

[20:28:05] Sally leaves the room

[20:30:11] snowstorm leaves the room

[20:34:44] ff leaves the room

[20:36:11] red curry leaves the room

[20:36:23] Kris joins the room

[20:38:32] Сергій leaves the room

[20:40:07] writer77 leaves the room

[20:40:07] Neimi leaves the room

[20:41:06] writer77 joins the room

[20:41:06] Neimi joins the room

[20:41:22] mightyBroccoli joins the room

[20:42:27] savagepeanut leaves the room

[20:43:45] p42ity leaves the room

[20:43:48] savagepeanut leaves the room

[20:43:58] savagepeanut joins the room

[20:44:21] ukko joins the room

[20:44:35] ukko leaves the room

[20:45:53] savagepeanut joins the room

[20:47:48] p42ity joins the room

[20:48:59] fabio leaves the room

[20:49:31] rom1dep leaves the room

[20:49:33] Herbert leaves the room

[20:49:33] Herbert joins the room

[20:52:33] fabio joins the room

[20:55:34] Сергій leaves the room

[20:55:54] p42ity leaves the room

[20:56:05] jayteeuk leaves the room: Stream closed by us: Replaced by new connection (conflict)

[20:56:06] jayteeuk joins the room

[20:56:27] debacle joins the room

[20:57:09] paul joins the room

[20:58:01] stp joins the room

[20:58:04] Squeaky Latex Folf joins the room

[20:59:36] Emru leaves the room

[21:01:45] jayteeuk leaves the room: Connection failed: connection closed

[21:02:22] jayteeuk joins the room

[21:04:39] stp leaves the room

[21:04:47] EmleyMoor leaves the room

[21:05:30] Tim joins the room

[21:05:49] EmleyMoor joins the room

[21:06:47] Citlali leaves the room

[21:07:01] rain leaves the room

[21:09:00] paul leaves the room

[21:09:20] rain joins the room

[21:09:20] carlosr leaves the room

[21:11:44] pod leaves the room

[21:12:03] hexZoN3 leaves the room

[21:12:26] danielepiccoli joins the room

[21:13:26] gooya leaves the room: Disconnected: Replaced by new connection

[21:13:37] gooya joins the room

[21:14:47] mikel leaves the room

[21:16:20] dusty joins the room

[21:16:27] dusty leaves the room

[21:17:00] rain leaves the room

[21:17:32] dusty joins the room

[21:17:37] dusty leaves the room

[21:18:32] neox leaves the room

[21:19:27] Titi leaves the room

[21:20:16] neox joins the room

[21:20:34] Titi joins the room

[21:20:49] dusty joins the room

[21:21:11] rain joins the room

[21:21:36] LNJ joins the room

[21:26:18] Titi leaves the room

[21:26:31] random_tester_123 leaves the room

[21:27:02] sander joins the room

[21:27:11] TheCoffeMaker leaves the room

[21:27:30] marc36 leaves the room

[21:27:31] TheCoffeMaker joins the room

[21:28:23] rozzin leaves the room

[21:28:31] opinionplatform.org_3 leaves the room

[21:28:44] leke joins the room

[21:29:44] opinionplatform.org_3 joins the room

[21:30:58] sander leaves the room

[21:31:02] sander joins the room

[21:31:33] opinionplatform.org_3 leaves the room

[21:31:58] danielepiccoli leaves the room

[21:32:04] opinionplatform.org_3 joins the room

[21:35:18] leke leaves the room

[21:35:38] deimosBSD leaves the room

[21:36:11] Citlali joins the room

[21:36:29] claas leaves the room

[21:36:32] Marc leaves the room

[21:36:47] pod leaves the room

[21:36:50] claas joins the room

[21:37:53] cel joins the room

[21:38:36] Marc joins the room

[21:39:36] camille-dupont joins the room

[21:45:18] p42ity joins the room

[21:45:35] mirux leaves the room

[21:46:13] p42ity leaves the room

[21:47:21] samuel leaves the room

[21:47:58] polymath leaves the room

[21:49:39] Holger leaves the room

[21:49:52] Holger joins the room

[21:50:24] polymath joins the room

[21:51:30] pitchum leaves the room

[21:51:35] LNJ leaves the room

[21:52:52] marc36 leaves the room

[21:53:10] mightyBroccoli leaves the room

[21:53:21] kaptan leaves the room

[21:54:36] sander leaves the room

[21:54:49] sander joins the room

[21:55:10] fabio leaves the room

[21:56:03] samuel joins the room

[21:59:11] APach leaves the room

[22:00:09] uira joins the room

[22:00:21] karme leaves the room

[22:05:37] leke joins the room

[22:07:47] akshay joins the room

[22:07:56] mike leaves the room

[22:08:40] mike joins the room

[22:09:18] fabio joins the room

[22:10:14] tobiasd leaves the room

[22:10:54] samuel leaves the room

[22:11:32] rossen leaves the room

[22:13:17] isotelo joins the room

[22:15:35] snowstorm joins the room

[22:17:07] cel leaves the room

[22:23:13] fabio leaves the room

[22:23:30] tobiasd leaves the room

[22:24:13] carlos leaves the room

[22:24:18] carlos joins the room

[22:24:36] fabio joins the room

[22:25:32] ghost leaves the room

[22:27:46] leke leaves the room

[22:29:52] Olivia leaves the room

[22:31:19] Titi leaves the room

[22:32:27] akshay leaves the room

[22:34:37] LNJ joins the room

[22:36:21] Сергій joins the room

[22:39:08] Ramiro Romani leaves the room

[22:39:20] LNJ leaves the room

[22:39:22] LNJ joins the room

[22:39:55] mike_ leaves the room

[22:40:33] mako joins the room

[22:40:49] 窃火者 leaves the room

[22:42:23] debacle leaves the room

[22:42:24] polymath leaves the room

[22:42:33] Ramiro Romani joins the room

[22:43:16] polymath joins the room

[22:45:36] rain leaves the room

[22:45:57] opinionplatform.org_3 leaves the room

[22:46:46] rain joins the room

[22:48:10] karme leaves the room

[22:49:08] polymath leaves the room

[22:50:55] mike leaves the room

[22:52:26] mike joins the room

[22:52:55] rintinfinn leaves the room

[22:53:36] rain leaves the room

[22:53:44] polymath joins the room

[22:54:25] gooya leaves the room

[22:54:40] rozzin joins the room

[22:55:09] isotelo leaves the room

[22:58:16] badlop leaves the room

[22:58:26] rain joins the room

[22:59:50] Tim leaves the room

[23:00:53] 窃火者 joins the room

[23:06:29] erik leaves the room

[23:07:00] Marc leaves the room

[23:07:11] rain leaves the room

[23:07:27] ukko joins the room

[23:08:03] Marc joins the room

[23:11:43] rain joins the room

[23:12:01] polymath leaves the room

[23:13:11] polymath joins the room

[23:13:58] opinionplatform.org_3 joins the room

[23:14:43] neox leaves the room

[23:15:12] inkey leaves the room

[23:15:12] inkey joins the room

[23:17:26] rozzin leaves the room

[23:18:17] rozzin joins the room

[23:20:33] Ramiro Romani leaves the room

[23:24:55] rain joins the room

[23:29:04] Jeroen joins the room

[23:29:05] Jeroen leaves the room

[23:30:32] writer77 leaves the room

[23:31:29] rozzin leaves the room

[23:32:04] rozzin joins the room

[23:32:13] Jeroen joins the room

[23:32:14] Jeroen leaves the room

[23:32:25] gooya leaves the room

[23:32:31] gooya joins the room

[23:33:16] Olivia leaves the room

[23:34:47] maidlover leaves the room

[23:35:39] camille-dupont leaves the room

[23:37:28] OOS-Bob joins the room

[23:38:31] eevvoor leaves the room

[23:38:46] 窃火者 leaves the room

[23:38:47] Manis leaves the room

[23:38:54] eevvoor joins the room

[23:39:01] fabio leaves the room

[23:39:16] rozzin leaves the room

[23:39:37] kapad leaves the room

[23:40:14] 窃火者 joins the room

[23:41:25] eevvoor leaves the room

[23:43:28] SouL leaves the room

[23:43:41] OOS-Bob leaves the room

[23:45:30] fabio joins the room

[23:47:02] Mitex Leo joins the room

[23:48:23] 窃火者 leaves the room

[23:51:00] leke leaves the room

[23:59:56] leke joins the room

Powered by ejabberd - robust, scalable and extensible XMPP server