summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* rgw: don't send tail to gc if copying object to itselfwip-4776-bobtailYehuda Sadeh2013-04-222-2/+6
| | | | | | | | | Fixes: #4776 Backport: bobtail Need to make sure that when copying an object into itself we don't send the tail to the garbage collection. Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
* Merge pull request #210 from dalgaaf/wip-da-bobtail-pybindJosh Durgin2013-04-111-5/+6
|\ | | | | Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
| * rados.py: fix create_pool()Danny Al-Gaaf2013-04-091-5/+6
| | | | | | | | | | | | | | | | | | | | Call rados_pool_create_with_all() only if auid and crush_rule are set properly. In case only crush_rule is set call rados_pool_create_with_crush_rule() on librados, not the other way around. Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de> (cherry picked from commit 94a1f25e7230a700f06a2699c9c2b99ec1bf7144)
* | mon: Use _daemon version of argparse functionsDan Mick2013-04-094-6/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | Allow argparse functions to fail if no argument given by using special versions that avoid the default CLI behavior of "cerr/exit" Fixes: #4678 Signed-off-by: Dan Mick <dan.mick@inktank.com> Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com> (cherry picked from commit be801f6c506d9fbfb6c06afe94663abdb0037be5) Conflicts: src/mon/Monitor.cc
* | ceph_argparse: add _daemon versions of argparse callsDan Mick2013-04-092-7/+50
|/ | | | | | | | | | | | | | mon needs to call argparse for a couple of -- options, and the argparse_witharg routines were attempting to cerr/exit on missing arguments. This is appropriate for the CLI usage, but not the daemon usage. Add a 'cli' flag that can be set false for the daemon usage (and cause the parsing routine to return false instead of exit). The daemon's parsing code due for a rewrite soon. Signed-off-by: Dan Mick <dan.mick@inktank.com> Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com> (cherry picked from commit c76bbc2e6df16d283cac3613628a44937e38bed8)
* silence logrotate some moreAlexandre Oliva2013-03-291-3/+3
| | | | | | | | | | | | I was getting email with logrotate error output from “which invoke-rc.d” on systems without an invoke-rc.d. This patch silences it. Silence stderr from which when running logrotate From: Alexandre Oliva <oliva@gnu.org> Signed-off-by: Alexandre Oliva <oliva@gnu.org> (cherry picked from commit d02340d90c9d30d44c962bea7171db3fe3bfba8e)
* Merge remote-tracking branch 'upstream/bobtail-4556' into bobtailSamuel Just2013-03-293-2/+16
|\ | | | | | | Reviewed-by: Samuel Just <sam.just@inktank.com>
| * osd: disallow classes with flags==0Sage Weil2013-03-271-0/+5
| | | | | | | | | | | | | | They must be RD, WR, or something.... Signed-off-by: Sage Weil <sage@inktank.com> (cherry picked from commit 89c69016e1dddb9f3ca40fd699e4a995ef1e3eee)
| * osd: EINVAL when rmw_flags is 0Sage Weil2013-03-271-0/+3
| | | | | | | | | | | | | | | | | | A broken client (e.g., v0.56) can send a request that ends up with an rmw_flags of 0. Treat this as invalid and return EINVAL. Fixes: #4556 Signed-off-by: Sage Weil <sage@inktank.com> (cherry picked from commit f2dda43c9ed4fda9cfa87362514985ee79e0ae15)
| * osd: fix detection of non-existent class methodSage Weil2013-03-272-1/+8
| | | | | | | | | | Signed-off-by: Sage Weil <sage@inktank.com> (cherry picked from commit 50b831e3641c21cd5b145271688189e199f432d1)
| * osd: tolerate rmw_flags==0Sage Weil2013-03-271-1/+0
| | | | | | | | | | | | | | | | We will let OSD return a proper error instead of asserting. This is effectively a backport of c313423cfda55a2231e000cd5ff20729310867f8. Signed-off-by: Sage Weil <sage@inktank.com>
* | OSD: always activate_map in advance_pgs, only send messages if upSamuel Just2013-03-291-5/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We should always handle_activate_map() after handle_advance_map() in order to kick the pg into a valid peering state for processing requests prior to dropping the lock. Additionally, we would prefer to avoid sending irrelevant messages during boot, so only send if we are up according to the current service osdmap. Fixes: #4572 Backport: bobtail Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Sage Weil <sage@inktank.com> (cherry picked from commit 4dfcad44431855ba7d68a1ccb41dc3cb5db6bb50)
* | PG: update PGPool::name in PGPool::updateSamuel Just2013-03-281-0/+1
| | | | | | | | | | | | | | Fixes: #4471 Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Sage Weil <sage@inktank.com> (cherry picked from commit f804892d725cfa25c242bdc577b12ee81dcc0dcc)
* | ReplicatedPG: send entire stats on OP_BACKFILL_FINISHSamuel Just2013-03-282-8/+29
|/ | | | | | | | | | | | | | Otherwise, we update the stat.stat structure, but not the stat.invalid_stats part. This will result in a recently split primary propogating the invalid stats but not the invalid marker. Sending the whole pg_stat_t structure also mirrors MOSDSubOp. Fixes: #4557 Backport: bobtail Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Sage Weil <sage@inktank.com> (cherry picked from commit 76b296f01fd0d337c8fc9f79013883e62146f0c6)
* test_librbd_fsx: fix image closingJosh Durgin2013-03-271-1/+10
| | | | | | | | | Always close the image we opened in check_clone(), and check the return code of the rbd_close() called before cloning. Refs: #3958 Signed-off-by: Josh Durgin <josh.durgin@inktank.com> (cherry picked from commit 94ae72546507799667197fd941633bb1fd2520c2)
* rbd: remove fiemap use from importJosh Durgin2013-03-251-186/+78
| | | | | | | | | | | | | | | On some kernels and filesystems fiemap can be racy and provide incorrect data even after an fsync. Later we can use SEEK_HOLE and SEEK_DATA, but for now just detect zero runs like we do with stdin. Basically this adapts import from stdin to work in the case of a file or block device, and gets rid of other cruft in the import that used fiemap. Fixes: #4388 Backport: bobtail Signed-off-by: Josh Durgin <josh.durgin@inktank.com> (cherry picked from commit 3091283895e8ffa3e4bda13399318a6e720d498f)
* v0.56.4v0.56.4Gary Lowell2013-03-252-1/+7
|
* rgw: bucket index ops on system buckets shouldn't do anythingYehuda Sadeh2013-03-252-1/+7
| | | | | | | | | | | | | Fixes: #4508 Backport: bobtail On certain bucket index operations we didn't check whether the bucket was a system bucket, which caused the operations to fail. This triggered an error message on bucket removal operations. Signed-off-by: Yehuda Sadeh <yehuda@inktank.com> Reviewed-by: Greg Farnum <greg@inktank.com> (cherry picked from commit 70e0ee8ba955322832f0c366537ddf7a0288761e)
* systest: restrict list error acceptanceJosh Durgin2013-03-251-1/+1
| | | | | | | | Only ignore errors after the midway point if the midway_sem_post is defined. Signed-off-by: Josh Durgin <josh.durgin@inktank.com> (cherry picked from commit 5b24a68b6e7d57bac688021b822fb2f73494c3e9)
* systest: fix race with pool deletionJosh Durgin2013-03-254-13/+19
| | | | | | | | | | | | The second test have pool deletion and object listing wait on the same semaphore to connect and start. This led to errors sometimes when the pool was deleted before it could be opened by the listing process. Add another semaphore so the pool deletion happens only after the listing has begun. Fixes: #4147 Signed-off-by: Josh Durgin <josh.durgin@inktank.com> (cherry picked from commit b0271e390564119e998e18189282252d54f75eb6)
* os/FileJournal: fix aio self-throttling deadlockSage Weil2013-03-221-8/+17
| | | | | | | | | | | | | | | | | This block of code tries to limit the number of aios in flight by waiting for the amount of data to be written to grow relative to a function of the number of aios. Strictly speaking, the condition we are waiting for is a function of both aio_num and the write queue, but we are only woken by changes in aio_num, and were (in rare cases) waiting when aio_num == 0 and there was no possibility of being woken. Fix this by verifying that aio_num > 0, and restructuring the loop to recheck that condition on each wakeup. Fixes: #4079 Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Samuel Just <sam.just@inktank.com> (cherry picked from commit e5940da9a534821d0d8f872c13f9ac26fb05a0f5)
* common/MemoryModel: remove logging to /tmp/memlogSage Weil2013-03-221-13/+1
| | | | | | | | | | | | This was a hack for dev purposes ages ago; remove it. The predictable filename is a security issue. CVE-2013-1882 Reported-by: Michael Scherer <misc@zarb.org> Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Dan Mick <dan.mick@inktank.com> (cherry picked from commit c524e2e01da41ab5b6362c117939ea1efbd98095)
* init-ceph: clean up temp ceph.conf filename on exitSage Weil2013-03-221-0/+1
| | | | | | Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Dan Mick <dan.mick@inktank.com> (cherry picked from commit 6a7ad2eac1db6abca3d7edb23ca9b80751400a23)
* init-ceph: push temp conf file to a unique location on remote hostSage Weil2013-03-221-2/+3
| | | | | | | | | | | The predictable file name is a security problem. CVE-2013-1882 Reported-by: Michael Scherer <misc@zarb.org> Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Dan Mick <dan.mick@inktank.com> (cherry picked from commit 051734522fea92878dd8139f28ec4e6b01371ede)
* mkcephfs: make remote temp directory name uniqueSage Weil2013-03-221-1/+2
| | | | | | | | | | | The predictable file name is a security problem. CVE-2013-1882 Reported-by: Michael Scherer <misc@zarb.org> Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Dan Mick <dan.mick@inktank.com> (cherry picked from commit f463ef78d77b11b5ad78b31e9a3a88d0a6e62bca)
* PG::GetMissing: need to check need_up_thru in MLogRec handlerSamuel Just2013-03-221-1/+8
| | | | | | | | Backport: bobtail Fixes: #4534 Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Sage Weil <sage@inktank.com> (cherry picked from commit 4fe4deafbe1758a6b3570048aca57485bd562440)
* PG,osd_types: improve check_new_interval debuggingSamuel Just2013-03-222-1/+15
| | | | | | | Backport: bobtail Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Sage Weil <sage@inktank.com> (cherry picked from commit d611eba9caf45f2d977c627b123462a073f523a4)
* FileStore: fix reversed collection_empty return valueSamuel Just2013-03-211-1/+1
| | | | | | | | | | | Backport: bobtail Fixes: #4380 Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Greg Farnum <greg@inktank.com> (cherry picked from commit 7a434d10da0f77e1b329de0b06b6645cd73cc81b) Conflicts: src/os/FileStore.cc
* FileStore: set replay guard on create_collectionSamuel Just2013-03-212-1/+26
| | | | | | | | | | | | | | | | | This should prevent sequences like: rmcoll a mkcoll a touch a foo <crash> from causing trouble by preventing the rmcoll and mkcoll from being replayed. Fixes: 4064 Backport: bobtail Signed-off-by: Samuel Just <sam.just@inktank.com> (cherry picked from commit 411770c45734c9827745ddc4018d86c14f2858a6)
* FileStore: _split_collection should not create the collectionSamuel Just2013-03-215-1/+69
| | | | | | | | This will simplify adding a replay guard to create_collection. Backport: bobtail Signed-off-by: Samuel Just <sam.just@inktank.com> (cherry picked from commit b184ff581a08c9e6ce5b858f06ccbe9d0e2a170b)
* client: use 4MB f_bsize and f_frsize for statfsSage Weil2013-03-211-5/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Old stat(1) reports: Block size: 1048576 Fundamental block size: 1048576 and the df(1) arithmetic works out. New stat(1) reports: Block size: 1048576 Fundamental block size: 4096 which is what we are shoving into statvfs, but we have the b_size and fr_size arithmetic swapped. However, doing the *correct* reporting would then break the old stat by making both sizes appear to be 4KB (or whatever). Sidestep the issue by making *both* values 4MB.. which is both large enough to report large FS sizes, and also the default stripe size and thus a "reasonable" value to report for a block size. Perhaps in the future, when we no longer care about old userland, we can report the page size for f_bsize, which is probably the "most correct" thing to do. Fixes: #3794. See also #3793. Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Greg Farnum <greg@inktank.com> (cherry picked from commit 7c94083643891c9d66a117352f312b268bdb1135)
* os/FileStore: check replay guard on src for collection renameSage Weil2013-03-201-0/+4
| | | | | | | | | | | | | | | | | | This avoids a problematic sequence like: - rename A/ -> B/ - remove B/1...100 - destroy B/ - create A/ - write A/101... <crash> - replay A/ -> B/ - remove B/1...100 (fails but tolerated) - destroy B/ (fails with ENOTEMPTY) Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Samuel Just <sam.just@inktank.com> (cherry picked from commit 5fc83c8d9887d2a916af11436ccc94fcbfe59b7a)
* PG::proc_replica_log: oinfo.last_complete must be *before* first entry in ↵Samuel Just2013-03-201-4/+17
| | | | | | | | omissing Fixes: #4189 Signed-off-by: Samuel Just <sam.just@inktank.com> (cherry picked from commit 2dae6a68ee85a20220ee940dbe33a2144d43457b)
* osd/PG: fix typo, missing -> omissingSage Weil2013-03-201-1/+1
| | | | | | | From ce7ffc34408bf32c66dc07e6f42d54b7ec489d41. Signed-off-by: Sage Weil <sage@inktank.com> (cherry picked from commit dc181224abf6fb8fc583730ae3d90acdf0b80f39)
* PG::proc_replica_log: adjust oinfo.last_complete based on omissingSamuel Just2013-03-201-0/+2
| | | | | | | | | | Otherwise, search_for_missing may neglect to check the missing set for some objects assuming that if the need version is prior to last_complete, the replica must have it. Fixes: #4994 Signed-off-by: Samuel Just <sam.just@inktank.com> (cherry picked from commit ce7ffc34408bf32c66dc07e6f42d54b7ec489d41)
* osd: fix load_pgs collection handlingSage Weil2013-03-191-28/+31
| | | | | | | | | | | | On a _TEMP pg, is_pg() would succeed, which meant we weren't actually hitting the cleanup checks. Instead, restructure this loop as positive checks and handle each type of collection we understand. This fixes _TEMP cleanup. Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Samuel Just <sam.just@inktank.com> (cherry picked from commit b19b6dced85617d594c15631571202aab2f94ae8)
* osd: fix load_pgs handling of pg dirs without a headSage Weil2013-03-191-0/+8
| | | | | | | | | | | | | If there is a pgid that passes coll_t::is_pg() but there is no head, we will populate the pgs map but then fail later when we try to do read_state. This is a side-effect of 55f8579. Take explicit note of _head collections we see, and then warn when we find stray snap collections. Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Samuel Just <sam.just@inktank.com> (cherry picked from commit 1f80a0b576c0af1931f743ad988b6293cbf2d6d9)
* OSD::load_pgs: first scan colls before initing PGsSamuel Just2013-03-191-1/+10
| | | | | | | Backport: bobtail Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Sage Weil <sage@inktank.com> (cherry picked from commit 073f58ede2e473af91f76d01679631c169274af7)
* osd: Add digest of omap for deep-scrubDavid Zafman2013-03-193-9/+50
| | | | | | | | | | | Add ScrubMap encode/decode v4 message with omap digest Compute digest of header and key/value. Use bufferlist to reflect structure and compute as we go, clearing bufferlist to reduce memory usage. Signed-off-by: David Zafman <david.zafman@inktank.com> Reviewed-by: Samuel Just <sam.just@inktank.com> (cherry picked from commit 509a93e89f04d7e9393090563cf7be8e0ea53891)
* OSD: split temp collection as wellSamuel Just2013-03-191-0/+9
| | | | | | | | | | | | | | Otherwise, when we eventually remove the temp collection, there might be objects in the temp collection which were independently pulled into the child pg collection. Thus, removing the old stale parent link from its temp collection also blasts the omap entries and snap mappings for the real child object. Backport: bobtail Fixes: #4452 Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Josh Durgin <josh.durgin@inktank.com> (cherry picked from commit f8d66e87a5c155b027cc6249006b83b4ac9b6c9b)
* PG: ignore non MISSING pg query in ReplicaActiveSamuel Just2013-03-191-3/+4
| | | | | | | | | | | | | | | | | | | | 1) Replica sends notify 2) Prior to processing notify, primary queues query to replica 3) Primary processes notify and activates sending MOSDPGLog to replica. 4) Primary does do_notifies at end of process_peering_events and sends to Query. 5) Replica sees MOSDPGLog and activates 6) Replica sees Query and asserts. In the above case, the Replica should simply ignore the old Query. Fixes: #4050 Backport: bobtail Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Sage Weil <sage@inktank.com> (cherry picked from commit 8222cbc8f35c359a35f8381ad90ff0eed5615dac)
* FileJournal: queue_pos \in [get_top(), header.max_size)Samuel Just2013-03-191-1/+1
| | | | | | | | | | | | If queue_pos == header.max_size when we create the entry header magic, the entry will be rejected at get_top() on replay. Fixes: #4436 Backport: bobtail Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Sage Weil <sage@inktank.com> (cherry picked from commit de8edb732e3a5ce4471670e43cfe6357ae6a2758)
* OSD: expand_pg_num after pg removesSamuel Just2013-03-191-5/+8
| | | | | | | | | | | | | | | | | | Otherwise: 1) expand_pg_num removes a splitting pg entry 2) peering thread grabs pg lock and starts split 3) OSD::consume_map grabs pg lock and starts removal At step 2), we run afoul of the assert(is_splitting) check in split_pgs. This way, the would be splitting pg is marked as removed prior to the splitting state being updated. Backport: bobtail Fixes: #4449 Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Sage Weil <sage@inktank.com> (cherry picked from commit f1b031b3cf195cf6df3d3c47c7d606fba63ed4c4)
* osd: update snap collections for sub_op_modify log records conditionalySage Weil2013-03-192-4/+7
| | | | | | | | | | | | | | | The only remaining caller is sub_op_modify(). If we do have a non-empty op transaction, we want to do this update, regardless of what we think last_backfill is (our notion may be not completely in sync with the primary). In particular, our last_backfill may be the same object but a different snapid, but the primary disagrees and is pushing an op transaction through. Instead, update the collections if we have a non-empty transaction. Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Samuel Just <sam.just@inktank.com> (cherry picked from commit 31e911b63d326bdd06981ec4029ad71b7479ed70)
* osd: include snaps in pg_log_entry_t::dump()Sage Weil2013-03-191-0/+14
| | | | | | Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Samuel Just <sam.just@inktank.com> (cherry picked from commit 715d8717a0e8a08fbe97a3e7d3ffd33aa9529d90)
* osd: unconditionally encode snaps bufferSage Weil2013-03-191-7/+7
| | | | | | | | | | | | | | Previously we would only encode the updated snaps vector for CLONE ops. This doesn't work for MODIFY ops generated by the snap trimmer, which may also adjust the clone collections. It is also possible that other operations may need to populate this field in the future (e.g., LOST_REVERT may, although it currently does not). Fixes: #4071, and possibly #4051. Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Samuel Just <sam.just@inktank.com> (cherry picked from commit 54b6dd924fea3af982f3d729150b6449f318daf2)
* osd: improve debug output on snap collectionsSage Weil2013-03-192-3/+15
| | | | | | Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Samuel Just <sam.just@inktank.com> (cherry picked from commit 8b05492ca5f1479589bb19c1ce058b0d0988b74f)
* PG: check_recovery_sources must happen even if not activeSamuel Just2013-03-152-4/+11
| | | | | | | | | | | | | | | | | | missing_loc/missing_loc_sources also must be cleaned up if a peer goes down during peering: 1) pg is in GetInfo, acting is [3,1] 2) we find object A on osd [0] in GetInfo 3) 0 goes down, no new peering interval since it is neither up nor acting, but peer_missing[0] is removed. 4) pg goes active and try to pull A from 0 since missing_loc did not get cleaned up. Backport: bobtail Fixes: #4371 Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Greg Farnum <greg@inktank.com> (cherry picked from commit de22b186c497ce151217aecf17a8d35cdbf549bb)
* HashIndex: _collection_list_partial must tolerate NULL nextSamuel Just2013-03-151-0/+3
| | | | | | | | Backport: bobtail Fixes: #4379 Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Greg Farnum <greg@inktank.com> (cherry picked from commit ce4432adc67dc2fc06dd21ea08e59d179496bcc6)
* OSD: lock not needed in ~DeletingState()Samuel Just2013-03-151-1/+0
| | | | | | | | | | | | No further refs to the object can remain at this point. Furthermore, the callbacks might lock mutexes of their own. Backport: bobtail Fixes: #4378 Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Greg Farnum <greg@inktank.com> (cherry picked from commit e4bf1bcab159d7c5b720f5da01877c0f67c16d16)
View Lorry Controller Status