| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The once-upon-a-time unique O_LAZY value I chose forever ago is now
O_NOATIME, which means that some clients are choosing relaxed
consistency without meaning to.
It is highly unlikely that a real O_LAZY will ever exist, and we can
select it in the ceph case with the ioctl or libcephfs call, so drop
any support for doing this via open(2) flags.
Update doc/lazy_posix.txt file re: lazy io.
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit 94afedf02d07ad4678222aa66289a74b87768810)
|
|
|
|
|
|
| |
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 69a55445439fce0dd6a3d32ff4bf436da42f1b11)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It is possible to start a sync when our newest monmap is 0. Usually we see
e0 from probe, but that isn't always published as part of the very first
paxos transaction due to the way PaxosService::_active generates it's
first initial commit.
In any case, having e0 here is harmless.
Fixes: #5509
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
(cherry picked from commit 85a1d6cc5d3852c94d1287b566656c5b5024fa13)
|
|
|
|
|
|
|
|
|
| |
256 is not as large as the upstream 512 MB, but will help signficiantly and
be less disruptive for existing cuttlefish clusters.
Sort-of backport of e93730b7ffa48b53c8da2f439a60cb6805facf5a.
Signed-off-by: Sage Weil <sage@inktank.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
108000 is about 3 hours if paxos is going full-bore (1 proposal/second).
That ought to be pretty safe. Otherwise, we start trimming to soon and a
slow sync will just have to restart when it finishes.
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
(cherry picked from commit 71ebfe7e1abe4795b46cf00dfe1b03d1893368b0)
Conflicts:
src/common/config_opts.h
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
level doesn't seem to like this when it races with an internal compaction
attempt (see below). Instead, let the store get opened by the ceph_mon
caller, and pull a bit of the logic into the caller to make the flow a
little easier to follow.
-2> 2013-06-25 17:49:25.184490 7f4d439f8780 10 needs_conversion
-1> 2013-06-25 17:49:25.184495 7f4d4065c700 5 asok(0x13b1460) entry start
0> 2013-06-25 17:49:25.316908 7f4d3fe5b700 -1 *** Caught signal (Segmentation fault) **
in thread 7f4d3fe5b700
ceph version 0.64-667-g089cba8 (089cba8fc0e8ae8aef9a3111cba7342ecd0f8314)
1: ceph-mon() [0x649f0a]
2: (()+0xfcb0) [0x7f4d435dccb0]
3: (leveldb::Table::BlockReader(void*, leveldb::ReadOptions const&, leveldb::Slice const&)+0x154) [0x806e54]
4: ceph-mon() [0x808840]
5: ceph-mon() [0x808b39]
6: ceph-mon() [0x806540]
7: (leveldb::DBImpl::DoCompactionWork(leveldb::DBImpl::CompactionState*)+0xdd) [0x7f363d]
8: (leveldb::DBImpl::BackgroundCompaction()+0x2c0) [0x7f4210]
9: (leveldb::DBImpl::BackgroundCall()+0x68) [0x7f4cc8]
10: ceph-mon() [0x80b3af]
11: (()+0x7e9a) [0x7f4d435d4e9a]
12: (clone()+0x6d) [0x7f4d4196bccd]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit ea1f316e5de21487ae034a1aa929068ba23ac525)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Match a symlink to a dir, not just dirs. This fixes the osd case of e.g.,
creating an osd in /data/osd$id in which ceph-disk makes a symlink from
/var/lib/ceph/osd/ceph-$id.
Fix proposed by Matt Thompson <matt.thompson@mandiant.com>; extended to
include the upstart users too.
Fixes: #5490
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
(cherry picked from commit 87c98e92d1375c8bc76196bbbf06f677bef95e64)
|
|
|
|
|
|
|
|
|
|
|
| |
Valgrind complains about an invalid read when we don't pad the allocation,
and because it is inlined we can't whitelist it for valgrind. Workaround
the warning by just padding our allocations a bit.
Fixes: #5346
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 49ff63b1750789070a8c6fef830c9526ae0f6d9f)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When there are more than one active MDS, restarting MDS triggers
assertion "reconnected_snaprealms.empty()" quite often. If there
is no snapshot in the FS, the items left in reconnected_snaprealms
should be other MDS' mdsdir. I think it's harmless.
If there are snapshots in the FS, the assertion probably can catch
real bugs. But at present, snapshot feature is broken, fixing it is
non-trivial. So replace the assertion with a warning.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
(cherry picked from commit 26effc0e583b0a3dade6ec81ef26dec1c94ac8b2)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We do two things here:
- make init an one-time unconditional init method, which is what the
health service expects/needs.
- switch PGMonitor::init to be post_paxos_update() which is called after
the other services update, which is what PGMonitor really needs.
This is a new version of the fix originally in commit
a2fe0137946541e7b3b537698e1865fbce974ca6 (and those around it). That is,
this re-fixes a problem where osds do not see pg creates from their
subscribe due to map_pg_creates() not getting called.
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit e635c47851d185eda557e36bdc4bf3775f7b87a2)
Conflicts:
src/mon/PGMonitor.cc
src/mon/PGMonitor.h
|
|
|
|
|
|
|
|
|
|
| |
Some services need to update internal state based on other service's
state, and thus need to be run after everyone has pulled their info out of
paxos.
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 131686980f0a930d5de7cbce8234fead5bd438b6)
|
|
|
|
|
|
|
| |
Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Joao Luis <joao.luis@inktank.com>
(cherry picked from commit bd8255a750de08c1b8ee5e9c9a0a1b9b16171462)
(cherry picked from commit 9e604ee6943fdb131978afbec51321050faddfc6)
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes: #5455
Backport: cuttlefish
This commit fixes a regression, where radosgw-admin buckets list
operation wasn't returning any data.
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
(cherry picked from commit e1f9fe58d2860fcbb18c92d3eb3946236b49a6ce)
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The lockfile class relies on file system trickery to get safe mutual
exclusion. However, the unix syscalls do this for us. More
importantly, the unix locks go away when the owning process dies, which
is behavior that we want here.
Fixes: #5387
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
(cherry picked from commit 2a4953b697a3464862fd3913336edfd7eede2487)
|
|
|
|
|
|
|
|
|
|
|
|
| |
If we see a 'ready' file in the target OSD dir, do not mount our device
on top of it.
Among other things, this prevents ceph-disk activate on stray disks from
stepping on teuthology osds.
Fixes: #5445
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 8a17f33b14d858235dfeaa42be1f4842dcfd66d2)
|
|
|
|
|
|
|
|
| |
If the underwater dentry is a remove link, we shouldn't mark the
inode clean
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
(cherry picked from commit 81d073fecb58e2294df12b71351321e6d2e69652)
|
|
|
|
|
|
|
|
|
|
| |
If we short-circuit and bootstrap, cancel our timer. Otherwise it will
go off some time later when we are in who knows what state.
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
(cherry picked from commit 9ae0ec83dabe37ac15e5165559debdfef7a5f91d)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If we are probing and get (say) an election timeout that calls reset(),
cancel the timer. Otherwise, we assert later with a splat like
2013-06-24 01:09:33.675882 7fb9627e7700 4 mon.b@0(leader) e1 probe_timeout 0x307a520
2013-06-24 01:09:33.676956 7fb9627e7700 -1 mon/Monitor.cc: In function 'void Monitor::probe_timeout(int)' thread 7fb9627e7700 time 2013-06-24 01:09:43.675904
mon/Monitor.cc: 1888: FAILED assert(is_probing() || is_synchronizing())
ceph version 0.64-613-g134d08a (134d08a9654f66634b893d493e4a92f38acc63cf)
1: (Monitor::probe_timeout(int)+0x161) [0x56f5c1]
2: (Context::complete(int)+0xa) [0x574a2a]
3: (SafeTimer::timer_thread()+0x425) [0x7059a5]
4: (SafeTimerThread::entry()+0xd) [0x7065dd]
5: (()+0x7e9a) [0x7fb966f62e9a]
6: (clone()+0x6d) [0x7fb9652f9ccd]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
Fixes: #5438
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
(cherry picked from commit 03d3be3eaa96a8e72754c36abd6f355c68d52d59)
|
|
|
|
|
|
|
|
|
|
|
|
| |
When you get device names like sdaa you do not want to mistakenly conclude that
sdaa is a partition of sda. Use /sys/block/$device/$partition existence
instead.
Fixes: #5211
Backport: cuttlefish
Signed-off-by: Alexandre Maragone <alexandre.maragone@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 8c0daafe003935881c5192e0b6b59b949269e5ae)
|
|
|
|
|
|
|
|
|
|
|
| |
client/Client.cc: In member function 'virtual void Client::ms_handle_remote_reset(Connection*)':
warning: client/Client.cc:7892:9: enumeration value 'STATE_NEW' not handled in switch [-Wswitch]
warning: client/Client.cc:7892:9: enumeration value 'STATE_OPEN' not handled in switch [-Wswitch]
warning: client/Client.cc:7892:9: enumeration value 'STATE_CLOSED' not handled in switch [-Wswitch]
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>
(cherry picked from commit 8bd936f077530dfeb2e699164e4492b1c0973088)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
create_initial called 2x
The create_initial() method may get called multiple times; make sure it
will unconditionally generate new/initial rotating keys. Move the block
up so that we can easily assert as much.
Broken by commit cd98eb0c651d9ee62e19c2cc92eadae9bed678cd.
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>
(cherry picked from commit 521fdc2a4e65559b3da83283e6ca607b6e55406f)
|
|
|
|
|
|
| |
Fixes: #5443
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 31d6062076fdbcd2691c07a23b381b26abc59f65)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ceph-mon recently started using Preforker to working around forking issues.
As a result, internal_safe_to_start_threads got set sooner and calls to
pick_addresses() which try to set string config values now fail because
there are no config observers for them.
Work around this by observing the change while we adjust the value. We
assume pick_addresses() callers are smart enough to realize that their
result will be reflected by cct->_conf and not magically handled elsewhere.
Fixes: #5195, #5205
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
(cherry picked from commit eb86eebe1ba42f04b46f7c3e3419b83eb6fe7f9a)
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In commit f985de28f86675e974ac7842a49922a35fe24c6c I mistakenly made
is_writeable() false while paxos was updating due to a misread of
Paxos::propose_new_value() (I didn't see that it would queue).
This is problematic because it narrows the window during which each service
is writeable for no reason.
Allow service to be writeable both when paxos is active and updating.
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 11169693d086e67dcf168ce65ef6e13eebd1a1ab)
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Treat this as an extension of the recovery process, e.g.
RECOVERING -> ACTIVE
or
RECOVERING -> UPDATING_PREVIOUS -> ACTIVE
and we are not active until we get to "the end" in both cases.
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 392a8e21f8571b410c85be2129ef62dd6fc52b54)
|
|
|
|
|
|
|
|
|
| |
- make states mutually exclusive (an enum)
- rename locked -> updating_previous
- set state prior to begin() to simplify things a bit
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit ee34a219605d1943740fdae0d84cfb9020302dd6)
|
|
|
|
|
|
|
|
|
|
|
| |
If we are re-proposing a previously accepted value from a previous quorum,
we should not consider it readable, because it is possible it was exposed
to clients as committed (2/3 accepted) but not recored to be committed, and
we do not want to expose old state as readable when new state was
previously readable.
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit ec2ea86ed55e00265c2cc5ad0c94460b4c92865c)
|
|
|
|
|
|
|
| |
This is never set when we block, and nobody looks at it.
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 7b7ea8e30e20704caad9a841332ecb2e39819a41)
|
|
|
|
|
|
|
|
| |
Recast this in terms of paxos check + our conditions, and make it
match wait_for_writeable().
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit f985de28f86675e974ac7842a49922a35fe24c6c)
|
|
|
|
|
|
|
|
| |
Recast this in terms of the paxos check and our additional conditions,
which match wait_for_readable().
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 3aa61a0beb540e48bf61ceded766d6ff52c95eb2)
|
|
|
|
|
| |
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit e832e76a4af04b091c806ad412bcfd0326d75a2d)
|
|
|
|
|
|
|
|
|
|
|
|
| |
The update_from_paxos() methods occasionally like to trigger new activity.
As long as they check is_readable() and is_writeable(), they will defer
until we go active and that activity will happen in the normal callbacks.
This fixes the problem where we active but is_writeable() is still false,
triggered by PGMonitor::check_osd_map().
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit e68b1bd36ed285e38a558899f83cf224d3aa60ed)
|
|
|
|
|
| |
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit dc83430124a5fd37573202a4cc0986c3c03739ef)
|
|
|
|
|
|
|
|
|
|
|
|
| |
Do the paxos refresh inside finish_proposal, ordered *after* the leader
assertion so that MonmapMonitor::update_from_paxos() calling bootstrap()
does not kill us.
Also, remove unnecessary finish_queued_proposal() and move the logic inline
where the bad leader assertion is obvious.
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit a42d7582f816b45f5d19c393fd45447555e78fdd)
|
|
|
|
|
|
|
|
| |
Refresh the in-memory values when we are told the on-disk paxos state
may have changed.
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
(cherry picked from commit 2fccb300bdf6ffd44db3462eb05115da11322ed4)
|
|
|
|
|
|
|
|
| |
The refresh is done explicitly by the monitor, independent of the more
fragile PaxosService callbacks.
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit d941363d6e4249e97b64faff0e573f75e918ac0c)
|
|
|
|
|
|
|
|
| |
The refresh() will do this when the state changes; no need to
opportunistically call this method all of the time.
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 03014a4ecc06cde420fad0c6c2a0177ebd7b839d)
|
|
|
|
|
|
|
|
|
| |
Instead of opportunistically calling each service's update_from_paxos(),
instead explicitly refresh all in-memory state whenever we know the
paxos state may have changed. This is simpler and less fragile.
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit cc339c07312006e65854207523f50542d00ecf87)
|
|
|
|
|
|
|
|
|
| |
This closes a very narrow race during mon creation where there are no
service keys.
Fixes: #5427
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit cd98eb0c651d9ee62e19c2cc92eadae9bed678cd)
|
|
|
|
|
|
|
|
|
|
| |
- Don't increment before we dereference!
- We need to update the iterator before we delete the item.
This code is changed in master, so this fix is for cuttlefish only.
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
|
|
|
|
|
|
|
|
| |
CID 1019628 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
2. uninit_member: Non-static class member "test_ops_hook" is not initialized in this constructor nor in any functions that it calls.
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit e30a03210c3efb768b1653df5ae58917ef26e579)
|
|
|
|
|
|
|
|
| |
CID 1019627 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member "next_notif_id" is not initialized in this constructor nor in any functions that it calls.
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 499edd8bfc355c2d590f5fa1ef197d1ea5680351)
|
|
|
|
|
| |
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit d60534b8f59798feaeeaa17adba2a417d7777cbf)
|
|
|
|
|
| |
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 8a4ed58e39b287fd8667c62b45848487515bdc80)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes valgrind warning:
==14803== Use of uninitialised value of size 8
==14803== at 0x12E7614: sctp_crc32c_sb8_64_bit (sctp_crc32.c:567)
==14803== by 0x12E76F8: update_crc32 (sctp_crc32.c:609)
==14803== by 0x12E7720: ceph_crc32c_le (sctp_crc32.c:733)
==14803== by 0x105085F: ceph::buffer::list::crc32c(unsigned int) (buffer.h:427)
==14803== by 0x115D7B2: Message::calc_front_crc() (Message.h:441)
==14803== by 0x1159BB0: Message::encode(unsigned long, bool) (Message.cc:170)
==14803== by 0x1323934: Pipe::writer() (Pipe.cc:1524)
==14803== by 0x13293D9: Pipe::Writer::entry() (Pipe.h:59)
==14803== by 0x120A398: Thread::_entry_func(void*) (Thread.cc:41)
==14803== by 0x503BE99: start_thread (pthread_create.c:308)
==14803== by 0x6C6E4BC: clone (clone.S:112)
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit eb91f41042fa31df2bef9140affa6eac726f6187)
|
|\
| |
| |
| | |
Reviewed-by: Samuel Just <sam.just@inktank.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
On XFS this call is problematic because it directly calls the filemap
writeback without vectoring through xfs. This can break the delicate
ordering of writeback and range zeroing; see #4976 and this thread
http://oss.sgi.com/archives/xfs/2013-06/msg00066.html
Drop this behavior for now to avoid subtle data corruption.
Signed-off-by: Sage Weil <sage@inktank.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The use of sync_file_range(2) on XFS screws up XFS' delicate ordering
of writeback and range zeroing; see #4976 and this thread:
http://oss.sgi.com/archives/xfs/2013-06/msg00066.html
Instead, replace all sync_file_range(2) calls with fdatasync(2), which
*does* do ordered writeback and should not leak unzeroed blocks.
Signed-off-by: Sage Weil <sage@inktank.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
common/Preforker.h: In member function ‘int Preforker::signal_exit(int)’:
warning: common/Preforker.h:82:45: ignoring return value of ‘ssize_t safe_write(int, const void*, size_t)’, declared with attribute warn_unused_result [-Wunused-result]
This is harder than it should be to fix. :(
http://stackoverflow.com/questions/3614691/casting-to-void-doesnt-remove-warn-unused-result-error
Whatever, I guess we can do something useful with this return value.
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>
(cherry picked from commit ce7b5ea7d5c30be32e4448ab0e7e6bb6147af548)
|