[OpenSIPS-Devel] [OpenSIPS/opensips] 0b3ad4: clusterer: Make the data sync interface more robust

Tue Jul 30 09:39:02 EDT 2019

  Branch: refs/heads/2.4
  Home:   https://github.com/OpenSIPS/opensips
  Commit: 0b3ad435be73d7ef49c511d2b70039c39a883135
      https://github.com/OpenSIPS/opensips/commit/0b3ad435be73d7ef49c511d2b70039c39a883135
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-07-29 (Mon, 29 Jul 2019)

  Changed paths:
    M modules/clusterer/sync.c

  Log Message:
  -----------
  clusterer: Make the data sync interface more robust

This patch improves the data sync interface so that during a sync,
modules are no longer forced to micro-manage the data packets they are
receiving from the interface.  They can now freely abort the processing
of a sync chunk at any time, without disrupting the processing of the
entire sync packet (composed of many more of such data chunks).

Additionally, since the sync packet format has changed (an extra integer
is needed for each chunk in order to allow the "skip" mechanism), the
sync packet version is now bumped from 1 -> 2, in order to prevent any
compatibility issues with OpenSIPS nodes without this patch.

  Commit: a20a0acb5d9e8ef75d4cf2bb081ed1d5d259a3dd
      https://github.com/OpenSIPS/opensips/commit/a20a0acb5d9e8ef75d4cf2bb081ed1d5d259a3dd
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-07-29 (Mon, 29 Jul 2019)

  Changed paths:
    M bin_interface.c
    M bin_interface.h
    M modules/clusterer/api.h
    M modules/clusterer/clusterer.h
    M modules/clusterer/sync.c
    M modules/clusterer/sync.h
    M modules/dialog/dlg_replication.c
    M modules/usrloc/ureplication.c

  Log Message:
  -----------
  clusterer: Enhance the versioning of sync packets

This commit adds an additional "version" field for the sync packets,
which are more complex than the other ones.  Since they contain
serialization logic from two different layers (clusterer + data module),
they should also contain two version fields, to allow each module to
discard data coming from an OpenSIPS donor node running on a differing
binary version.

  Commit: e33565342065298eeb542a73503f84402cc1076d
      https://github.com/OpenSIPS/opensips/commit/e33565342065298eeb542a73503f84402cc1076d
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-07-29 (Mon, 29 Jul 2019)

  Changed paths:
    M modules/dialog/dlg_replication.c

  Log Message:
  -----------
  dialog replication: Revert the sync packet alignment code

... since now it is unnecessary, thanks to the sync layer enhancements.

  Commit: b8bde2f014e5b427c8517493a7ada16923d0928c
      https://github.com/OpenSIPS/opensips/commit/b8bde2f014e5b427c8517493a7ada16923d0928c
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-07-29 (Mon, 29 Jul 2019)

  Changed paths:
    M modules/dialog/dlg_replication.c

  Log Message:
  -----------
  dialog sync: Do not include early or ended dialogs

  Commit: aa93d0fbf369078f1c0e85fc10314fe7799aeca0
      https://github.com/OpenSIPS/opensips/commit/aa93d0fbf369078f1c0e85fc10314fe7799aeca0
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-07-30 (Tue, 30 Jul 2019)

  Changed paths:
    M modules/dialog/dlg_db_handler.c
    M modules/dialog/dlg_hash.c
    M modules/dialog/dlg_hash.h
    M modules/dialog/dlg_replication.c

  Log Message:
  -----------
  dialog: Fix data reload race conditions on startup

Since loading the data on child_init(), the load_dialog_info_from_db()
routines and rcv_cluster_event() routines could run in parallel, without
any synchronization on the dialog table, which could lead to duplicate
dialogs in the hash.

  Commit: f88c41064c9d895e52c6a8dac62867246da3b253
      https://github.com/OpenSIPS/opensips/commit/f88c41064c9d895e52c6a8dac62867246da3b253
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-07-30 (Tue, 30 Jul 2019)

  Changed paths:
    M modules/dialog/dlg_db_handler.c
    M modules/dialog/dlg_handlers.c
    M modules/dialog/dlg_hash.h
    M modules/dialog/dlg_replication.c
    M modules/dialog/dlg_req_within.h

  Log Message:
  -----------
  dialog: Decrement dialog stats during post-sync cleanup

After a sync completes, it proceeds to clean up all dialogs loaded from
DB which did not match the data received via sync.  However, make sure
to also decrement the 'active' / 'early' dialog stats on each delete!

  Commit: 51b5ec3bac182104781a5e12287a217053170432
      https://github.com/OpenSIPS/opensips/commit/51b5ec3bac182104781a5e12287a217053170432
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-07-30 (Tue, 30 Jul 2019)

  Changed paths:
    M modules/dialog/dlg_replication.c

  Log Message:
  -----------
  dialog sync: Fix ref miscount during post-sync cleanup

The hash reference must only be decremented a single time during the
lifetime of a dialog.  Given that multiple pieces of code may attempt
to delete a dialog concurrently (e.g. a BIN "delete" packet and the
post-sync cleanup routine), the only way to guarantee a single decrement
of the hash ref is by using the dialog state machine transition.

Iff we're the ones to transition from ACK -> DELETED, we can (and MUST)
also decrement the hash reference.

  Commit: 65a9f51f1ee43e500d6fbe34c3a0e07722bd75b2
      https://github.com/OpenSIPS/opensips/commit/65a9f51f1ee43e500d6fbe34c3a0e07722bd75b2
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-07-30 (Tue, 30 Jul 2019)

  Changed paths:
    M modules/dialog/dlg_db_handler.c
    M modules/dialog/dlg_db_handler.h
    M modules/dialog/dlg_replication.c

  Log Message:
  -----------
  dialog: Fix broken re-INVITE pinging after failover

The mandatory re-INVITE pinging data (SDP1, SDP2, ct1, ct2) was not
included in the BIN replication packets, so the pinging would stop
working once we'd failover to the backup box, in an active/backup
HA scenario.

  Commit: 3b1c49fdf99461594919fd9b5cf361fdb6e44372
      https://github.com/OpenSIPS/opensips/commit/3b1c49fdf99461594919fd9b5cf361fdb6e44372
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-07-30 (Tue, 30 Jul 2019)

  Changed paths:
    M bin_interface.c
    M bin_interface.h
    M modules/clusterer/api.h
    M modules/clusterer/clusterer.h
    M modules/clusterer/sync.c
    M modules/clusterer/sync.h
    M modules/dialog/dlg_db_handler.c
    M modules/dialog/dlg_db_handler.h
    M modules/dialog/dlg_handlers.c
    M modules/dialog/dlg_hash.c
    M modules/dialog/dlg_hash.h
    M modules/dialog/dlg_replication.c
    M modules/dialog/dlg_req_within.h
    M modules/usrloc/ureplication.c

  Log Message:
  -----------
  Merge branch 'bugfix/fix-dialog-sync-bugs' into 2.4

Compare: https://github.com/OpenSIPS/opensips/compare/ade908fba2bf...3b1c49fdf994