osd: let pgs process map advances before booting

The OSD deliberate consumes and processes most OSDMaps from while it
was down before it marks itself up, as this is can be slow.  The new
threading code does this asynchronously in peering_wq, though, and
does not let it drain before booting the OSD.  The OSD can get into
a situation where it marks itself up but is not responsive or useful
because of the backlog, and only makes the situation works by
generating more osdmaps as result.

Fix this by calling activate_map() even when booting, and when booting
draining the peering_wq on each call.  This is harmless since we are
not yet processing actual ops; we only need to be async when active.

Fixes: #3714
Signed-off-by: Sage Weil <sage@inktank.com>

1 files changed, 9 insertions, 3 deletions

diff --git a/src/osd/OSD.cc b/src/osd/OSD.cc
index 44d64b8000e..93b0fc4f10b 100644
--- a/src/osd/OSD.cc
+++ b/src/osd/OSD.cc
@@ -3956,8 +3956,7 @@ void OSD::handle_osd_map(MOSDMap *m)
   check_osdmap_features();
 
   // yay!
-  if (is_active())
-    activate_map();
+  activate_map();
 
   if (m->newest_map && m->newest_map > last) {
     dout(10) << " msg say newest map is " << m->newest_map << ", requesting more" << dendl;
@@ -4048,7 +4047,8 @@ void OSD::advance_pg(
 
     lastmap = nextmap;
   }
-  pg->handle_activate_map(rctx);
+  if (!is_booting())
+    pg->handle_activate_map(rctx);
 }
 
 /** 
@@ -4207,6 +4207,12 @@ void OSD::activate_map()
   wake_all_pg_waiters();   // the pg mapping may have shifted
   maybe_update_heartbeat_peers();
 
+  if (!is_active()) {
+    dout(10) << " not yet active; waiting for peering wq to drain" << dendl;
+    peering_wq.drain();
+    return;
+  }
+
   if (osdmap->test_flag(CEPH_OSDMAP_FULL)) {
     dout(10) << " osdmap flagged full, doing onetime osdmap subscribe" << dendl;
     monc->sub_want("osdmap", osdmap->get_epoch() + 1, CEPH_SUBSCRIBE_ONETIME);