From 8769eef34069efc3ffcf138da164221458c329b3 Mon Sep 17 00:00:00 2001 From: Walmyr Date: Fri, 14 Jun 2019 05:07:29 +0000 Subject: Backport of EE changes from MR 13763 https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/13763 --- doc/development/testing_guide/end_to_end/quick_start_guide.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md index 521e3e56e7a..1802f4792e0 100644 --- a/doc/development/testing_guide/end_to_end/quick_start_guide.md +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -357,13 +357,13 @@ In the following we describe the changes needed in each of the resource files me Now, let's make it possible to create an issue resource through the API. -First, in the [issue resource](https://gitlab.com/gitlab-org/gitlab-ee/blob/d3584e80b4236acdf393d815d604801573af72cc/qa/qa/resource/issue.rb), let's expose its labels attribute. +First, in the [issue resource](https://gitlab.com/gitlab-org/gitlab-ee/blob/d3584e80b4236acdf393d815d604801573af72cc/qa/qa/resource/issue.rb), let's expose its id and labels attributes. -Add the following `attribute :labels` right above the [`attribute :title`](https://gitlab.com/gitlab-org/gitlab-ee/blob/d3584e80b4236acdf393d815d604801573af72cc/qa/qa/resource/issue.rb#L15). +Add the following `attribute :id` and `attribute :labels` right above the [`attribute :title`](https://gitlab.com/gitlab-org/gitlab-ee/blob/d3584e80b4236acdf393d815d604801573af72cc/qa/qa/resource/issue.rb#L15). -> This line is needed to allow for labels to be automatically added to an issue when fabricating it via API. +> This line is needed to allow for the issue fabrication, and for labels to be automatically added to the issue when fabricating it via API. -> We add the new line above the existing attribute to keep them alphabetically organized. +> We add the attributes above the existing attribute to keep them alphabetically organized. Next, add the following code right below the [`fabricate!`](https://gitlab.com/gitlab-org/gitlab-ee/blob/d3584e80b4236acdf393d815d604801573af72cc/qa/qa/resource/issue.rb#L27) method. -- cgit v1.2.1 From 657760ba527402ef7b29b7f25a6e5c2d06db29c7 Mon Sep 17 00:00:00 2001 From: Mark Lapierre Date: Mon, 17 Jun 2019 03:07:32 +0000 Subject: Fix list rendering on docs site --- doc/development/testing_guide/end_to_end/quick_start_guide.md | 1 + 1 file changed, 1 insertion(+) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md index 1802f4792e0..11375b9b489 100644 --- a/doc/development/testing_guide/end_to_end/quick_start_guide.md +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -532,6 +532,7 @@ end ##### Details of `select_labels_and_refresh` Notice that we have not only moved the `select_labels_and_refresh` method, but we have also changed its implementation to: + 1. Click the `:edit_link_labels` element previously defined, instead of using `find('.block.labels .edit-link').click` 2. Use `within_element(:dropdown_menu_labels, text: label)`, and inside of it, we call `send_keys_to_element(:dropdown_input_field, [label, :enter])`, which is a method that we will implement in the `QA::Page::Base` class to replace `find('.dropdown-menu-labels .dropdown-input-field').send_keys [label, :enter]` 3. Use `click_body` after iterating on each label, instead of using `find('#content-body').click` -- cgit v1.2.1 From 7bfe9ae2fbac3640b6642182f35e9fae6e67a5b1 Mon Sep 17 00:00:00 2001 From: Walmyr Date: Mon, 17 Jun 2019 12:13:25 +0000 Subject: Turn single line code into blocks To improve doc readability. Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/62506 --- .../testing_guide/end_to_end/quick_start_guide.md | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md index 1802f4792e0..a32a5bfb8d4 100644 --- a/doc/development/testing_guide/end_to_end/quick_start_guide.md +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -547,15 +547,27 @@ Now let's change the view and the `dropdowns_helper` files to add the selectors In the [app/views/shared/issuable/_sidebar.html.haml](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/app/views/shared/issuable/_sidebar.html.haml) file, on [line 105 ](https://gitlab.com/gitlab-org/gitlab-ee/blob/84043fa72ca7f83ae9cde48ad670e6d5d16501a3/app/views/shared/issuable/_sidebar.html.haml#L105), add an extra class `qa-edit-link-labels`. -The code should look like this: `= link_to _('Edit'), '#', class: 'js-sidebar-dropdown-toggle edit-link float-right qa-edit-link-labels'`. +The code should look like this: + +```haml += link_to _('Edit'), '#', class: 'js-sidebar-dropdown-toggle edit-link float-right qa-edit-link-labels' +``` In the same file, on [line 121](https://gitlab.com/gitlab-org/gitlab-ee/blob/84043fa72ca7f83ae9cde48ad670e6d5d16501a3/app/views/shared/issuable/_sidebar.html.haml#L121), add an extra class `.qa-dropdown-menu-labels`. -The code should look like this: `.dropdown-menu.dropdown-select.dropdown-menu-paging.dropdown-menu-labels.dropdown-menu-selectable.qa-dropdown-menu-labels`. +The code should look like this: + +```haml +.dropdown-menu.dropdown-select.dropdown-menu-paging.dropdown-menu-labels.dropdown-menu-selectable.qa-dropdown-menu-labels +``` In the [`dropdowns_helper.rb`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/app/helpers/dropdowns_helper.rb) file, on [line 94](https://gitlab.com/gitlab-org/gitlab-ee/blob/99e51a374f2c20bee0989cac802e4b5621f72714/app/helpers/dropdowns_helper.rb#L94), add an extra class `qa-dropdown-input-field`. -The code should look like this: `filter_output = search_field_tag search_id, nil, class: "dropdown-input-field qa-dropdown-input-field", placeholder: placeholder, autocomplete: 'off'`. +The code should look like this: + +```ruby +filter_output = search_field_tag search_id, nil, class: "dropdown-input-field qa-dropdown-input-field", placeholder: placeholder, autocomplete: 'off' +``` > Classes starting with `qa-` are used for testing purposes only, and by defining such classes in the elements we add **testability** in the application. -- cgit v1.2.1 From 09d7f0c0e0c6c7906078dbf1797b08e08d0e696c Mon Sep 17 00:00:00 2001 From: Mark Lapierre Date: Wed, 19 Jun 2019 00:22:23 +0000 Subject: Update docs on how to run E2E tests Make instructions on how to run the E2E tests against GDK more explicit and easier to find. --- doc/development/testing_guide/end_to_end/index.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md index afd81ff00b2..527cd350633 100644 --- a/doc/development/testing_guide/end_to_end/index.md +++ b/doc/development/testing_guide/end_to_end/index.md @@ -126,6 +126,18 @@ See [Review Apps][review-apps] for more details about Review Apps. [helm-chart]: https://gitlab.com/charts/gitlab/ [cng]: https://gitlab.com/gitlab-org/build/CNG +## How do I run the tests? + +There are two main options for running the tests. If you simply want to run the +existing tests against a live GitLab instance or against a pre-built docker image +you can use the [GitLab QA orchestrator][gitlab-qa-readme]. See also [examples +of the test scenarios you can run via the orchestrator](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/what_tests_can_be_run.md#examples). + +On the other hand, if you would like to run against a local development GitLab +environment, you can use the [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit/). +Please refer to the instructions in the [QA README](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/README.md#how-can-i-use-it) +and the section below. + ## How do I write tests? In order to write new tests, you first need to learn more about GitLab QA -- cgit v1.2.1 From 9d67a3d17ca5167ddab55009ad5fc7742a6b75cf Mon Sep 17 00:00:00 2001 From: rpereira2 Date: Wed, 19 Jun 2019 16:50:00 +0530 Subject: Change docs to recommend perform_enqueued_jobs - Instead of Sidekiq::Testing.inline!, recommend the use of perform_enqueued_jobs. This is due to the changes in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20768 --- doc/development/testing_guide/best_practices.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index 71e3b7740cb..448d9fd01c4 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -327,7 +327,7 @@ However, if a spec makes direct Redis calls, it should mark itself with the `:clean_gitlab_redis_queues` traits as appropriate. Sidekiq jobs are typically not run in specs, but this behaviour can be altered -in each spec through the use of `Sidekiq::Testing.inline!` blocks. Any spec that +in each spec through the use of `perform_enqueued_jobs` blocks. Any spec that causes Sidekiq jobs to be pushed to Redis should use the `:sidekiq` trait, to ensure that they are removed once the spec completes. -- cgit v1.2.1 From 02eece753967fa2071ca9edbd866376987d803a6 Mon Sep 17 00:00:00 2001 From: Walmyr Lima Date: Wed, 19 Jun 2019 16:01:25 +0200 Subject: Backport EE MR https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/14280 --- doc/development/testing_guide/end_to_end/quick_start_guide.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md index d33ef0fc229..f96c85be1ba 100644 --- a/doc/development/testing_guide/end_to_end/quick_start_guide.md +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -247,7 +247,7 @@ module QA [@new_label_same_scope, @new_label_different_scope].each do |label| Resource::Label.fabricate_via_api! do |l| - l.project = issue.project.id + l.project = issue.project l.title = label end end @@ -414,7 +414,7 @@ def api_get_path end def api_post_path - "/projects/#{project}/labels" + "/projects/#{project.id}/labels" end def api_post_body -- cgit v1.2.1 From e128239340010401b4e7fd391686dcc48c42fff0 Mon Sep 17 00:00:00 2001 From: Evan Read Date: Tue, 25 Jun 2019 04:34:55 +0000 Subject: Add Markdown linting Also adds and one linting rule and makes project conform to it. --- .../testing_guide/end_to_end/dynamic_element_validation.md | 12 ++++++------ doc/development/testing_guide/end_to_end/page_objects.md | 6 +++--- 2 files changed, 9 insertions(+), 9 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/dynamic_element_validation.md b/doc/development/testing_guide/end_to_end/dynamic_element_validation.md index f7b3ca8bc89..aec0a3ede5a 100644 --- a/doc/development/testing_guide/end_to_end/dynamic_element_validation.md +++ b/doc/development/testing_guide/end_to_end/dynamic_element_validation.md @@ -5,8 +5,8 @@ We devised a solution to solve common test automation problems such as the dread Other problems that dynamic element validations solve are... - When we perform an action with the mouse, we expect something to occur. -- When our test is navigating to (or from) a page, we ensure that we are on the page we expect before -test continuation. +- When our test is navigating to (or from) a page, we ensure that we are on the page we expect before + test continuation. ## How it works @@ -19,7 +19,7 @@ We interpret user actions on the page to have some sort of effect. These actions When a page is navigated to, there are elements that will always appear on the page unconditionally. -Dynamic element validation is instituted when using +Dynamic element validation is instituted when using ```ruby Runtime::Browser.visit(:gitlab, Some::Page) @@ -27,7 +27,7 @@ Runtime::Browser.visit(:gitlab, Some::Page) ### Clicks -When we perform a click within our tests, we expect something to occur. That something could be a component to now +When we perform a click within our tests, we expect something to occur. That something could be a component to now appear on the webpage, or the test to navigate away from the page entirely. Dynamic element validation is instituted when using @@ -71,7 +71,7 @@ class MyPage < Page::Base element :another_element, required: true element :conditional_element end - + def open_layer click_element :my_element, Layer::MyLayer end @@ -95,7 +95,7 @@ execute_stuff ``` will invoke GitLab QA to scan `MyPage` for `my_element` and `another_element` to be on the page before continuing to -`execute_stuff` +`execute_stuff` ### Clicking diff --git a/doc/development/testing_guide/end_to_end/page_objects.md b/doc/development/testing_guide/end_to_end/page_objects.md index 73e1fd862c1..05cb03eb4bd 100644 --- a/doc/development/testing_guide/end_to_end/page_objects.md +++ b/doc/development/testing_guide/end_to_end/page_objects.md @@ -82,7 +82,7 @@ module Page end # ... - end + end end end ``` @@ -134,7 +134,7 @@ for each element defined. In our case, `qa-login-field`, `qa-password-field` and `qa-sign-in-button` -**app/views/my/view.html.haml** +**app/views/my/view.html.haml** ```haml = f.text_field :login, class: "form-control top qa-login-field", autofocus: "autofocus", autocapitalize: "off", autocorrect: "off", required: true, title: "This field is required." @@ -146,7 +146,7 @@ Things to note: - The CSS class must be `kebab-cased` (separated with hyphens "`-`") - If the element appears on the page unconditionally, add `required: true` to the element. See -[Dynamic element validation](dynamic_element_validation.md) + [Dynamic element validation](dynamic_element_validation.md) ## Running the test locally -- cgit v1.2.1 From 4a125e97ccd9adf1007e20f3e7415d973d02486f Mon Sep 17 00:00:00 2001 From: Winnie Hellmann Date: Tue, 25 Jun 2019 15:28:33 +0000 Subject: Clarify that Jest does not use Webpack (docs) --- doc/development/testing_guide/frontend_testing.md | 2 ++ 1 file changed, 2 insertions(+) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index 4c9d1684c00..3949f61d4d4 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -28,6 +28,8 @@ we need to solve before being able to use Jest for all our needs. - Jest runs in a Node.js environment, not in a browser. Support for running Jest tests in a browser [is planned](https://gitlab.com/gitlab-org/gitlab-ce/issues/58205). - Because Jest runs in a Node.js environment, it uses [jsdom](https://github.com/jsdom/jsdom) by default. +- Jest does not have access to Webpack loaders or aliases. + The aliases used by Jest are defined in its [own config](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/jest.config.js). - All calls to `setTimeout` and `setInterval` are mocked away. See also [Jest Timer Mocks](https://jestjs.io/docs/en/timer-mocks). - `rewire` is not required because Jest supports mocking modules. See also [Manual Mocks](https://jestjs.io/docs/en/manual-mocks). - The following will cause tests to fail in Jest: -- cgit v1.2.1 From 1eccad79c1850fd047884406e0906c928533dc42 Mon Sep 17 00:00:00 2001 From: Winnie Hellmann Date: Tue, 25 Jun 2019 15:37:41 +0000 Subject: Clarify that this.something is not available in Jest (docs) --- doc/development/testing_guide/frontend_testing.md | 3 +++ 1 file changed, 3 insertions(+) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index 3949f61d4d4..fc9b175bc8a 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -32,6 +32,9 @@ we need to solve before being able to use Jest for all our needs. The aliases used by Jest are defined in its [own config](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/jest.config.js). - All calls to `setTimeout` and `setInterval` are mocked away. See also [Jest Timer Mocks](https://jestjs.io/docs/en/timer-mocks). - `rewire` is not required because Jest supports mocking modules. See also [Manual Mocks](https://jestjs.io/docs/en/manual-mocks). +- No [context object](https://jasmine.github.io/tutorials/your_first_suite#section-The_%3Ccode%3Ethis%3C/code%3E_keyword) is passed to tests in Jest. + This means sharing `this.something` between `beforeEach()` and `it()` for example does not work. + Instead you should declare shared variables in the context that they are needed (via `const` / `let`). - The following will cause tests to fail in Jest: - Unmocked requests. - Unhandled Promise rejections. -- cgit v1.2.1 From 6ca0f281fbeda04cede1ba9afae7df37eefbc609 Mon Sep 17 00:00:00 2001 From: Winnie Hellmann Date: Wed, 26 Jun 2019 11:39:30 +0000 Subject: Document limitations of jsdom --- doc/development/testing_guide/frontend_testing.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index fc9b175bc8a..28ebb6f0f64 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -27,7 +27,7 @@ we need to solve before being able to use Jest for all our needs. ### Differences to Karma - Jest runs in a Node.js environment, not in a browser. Support for running Jest tests in a browser [is planned](https://gitlab.com/gitlab-org/gitlab-ce/issues/58205). -- Because Jest runs in a Node.js environment, it uses [jsdom](https://github.com/jsdom/jsdom) by default. +- Because Jest runs in a Node.js environment, it uses [jsdom](https://github.com/jsdom/jsdom) by default. See also its [limitations](#limitations-of-jsdom) below. - Jest does not have access to Webpack loaders or aliases. The aliases used by Jest are defined in its [own config](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/jest.config.js). - All calls to `setTimeout` and `setInterval` are mocked away. See also [Jest Timer Mocks](https://jestjs.io/docs/en/timer-mocks). @@ -40,6 +40,17 @@ we need to solve before being able to use Jest for all our needs. - Unhandled Promise rejections. - Calls to `console.warn`, including warnings from libraries like Vue. +### Limitations of jsdom + +As mentioned [above](#differences-to-karma), Jest uses jsdom instead of a browser for running tests. +This comes with a number of limitations, namely: + +- [No scrolling support](https://github.com/jsdom/jsdom/blob/15.1.1/lib/jsdom/browser/Window.js#L623-L625) +- [No element sizes or positions](https://github.com/jsdom/jsdom/blob/15.1.1/lib/jsdom/living/nodes/Element-impl.js#L334-L371) +- [No layout engine](https://github.com/jsdom/jsdom/issues/1322) in general + +See also the issue for [support running Jest tests in browsers](https://gitlab.com/gitlab-org/gitlab-ce/issues/58205). + ### Debugging Jest tests Running `yarn jest-debug` will run Jest in debug mode, allowing you to debug/inspect as described in the [Jest docs](https://jestjs.io/docs/en/troubleshooting#tests-are-failing-and-you-don-t-know-why). -- cgit v1.2.1 From 4767dda56cccffe3686e3cc6e81ba9367f8c3f66 Mon Sep 17 00:00:00 2001 From: Walmyr Lima Date: Tue, 25 Jun 2019 19:38:45 +0200 Subject: Backport of EE MR https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/14320 --- .../testing_guide/end_to_end/quick_start_guide.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md index f96c85be1ba..670d2b31a29 100644 --- a/doc/development/testing_guide/end_to_end/quick_start_guide.md +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -242,7 +242,7 @@ module QA issue = Resource::Issue.fabricate_via_api! do |issue| issue.title = 'Issue to test the scoped labels' - issue.labels = @initial_label + issue.labels = [@initial_label] end [@new_label_same_scope, @new_label_different_scope].each do |label| @@ -365,6 +365,14 @@ Add the following `attribute :id` and `attribute :labels` right above the [`attr > We add the attributes above the existing attribute to keep them alphabetically organized. +Then, let's initialize an instance variable for labels to allow an empty array as default value when such information is not passed during the resource fabricatioin, since this optional. [Between the attributes and the `fabricate!` method](https://gitlab.com/gitlab-org/gitlab-ee/blob/1a1f1408728f19b2aa15887cd20bddab7e70c8bd/qa/qa/resource/issue.rb#L18), add the following: + +```ruby +def initialize + @labels = [] +end +``` + Next, add the following code right below the [`fabricate!`](https://gitlab.com/gitlab-org/gitlab-ee/blob/d3584e80b4236acdf393d815d604801573af72cc/qa/qa/resource/issue.rb#L27) method. ```ruby @@ -378,7 +386,7 @@ end def api_post_body { - labels: [labels], + labels: labels, title: title } end -- cgit v1.2.1 From e6257e43cb524156b8f116cdd0894bbc7a417be1 Mon Sep 17 00:00:00 2001 From: Walmyr Lima Date: Thu, 27 Jun 2019 17:18:18 +0200 Subject: Fix typo on end-to-end quick start guide --- doc/development/testing_guide/end_to_end/quick_start_guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md index 670d2b31a29..041bdf716b3 100644 --- a/doc/development/testing_guide/end_to_end/quick_start_guide.md +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -365,7 +365,7 @@ Add the following `attribute :id` and `attribute :labels` right above the [`attr > We add the attributes above the existing attribute to keep them alphabetically organized. -Then, let's initialize an instance variable for labels to allow an empty array as default value when such information is not passed during the resource fabricatioin, since this optional. [Between the attributes and the `fabricate!` method](https://gitlab.com/gitlab-org/gitlab-ee/blob/1a1f1408728f19b2aa15887cd20bddab7e70c8bd/qa/qa/resource/issue.rb#L18), add the following: +Then, let's initialize an instance variable for labels to allow an empty array as default value when such information is not passed during the resource fabrication, since this optional. [Between the attributes and the `fabricate!` method](https://gitlab.com/gitlab-org/gitlab-ee/blob/1a1f1408728f19b2aa15887cd20bddab7e70c8bd/qa/qa/resource/issue.rb#L18), add the following: ```ruby def initialize -- cgit v1.2.1 From 20654fb9bf7673cf4f925f36e7bdc4818233cbb9 Mon Sep 17 00:00:00 2001 From: Marcel Amirault Date: Mon, 1 Jul 2019 03:36:23 +0000 Subject: Enforce consistent prefix for bullet lists Adjusts asterisks to hyphens in assorted docs to allow lint rule to pass --- doc/development/testing_guide/review_apps.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md index 63b7b97c32f..ae40d628717 100644 --- a/doc/development/testing_guide/review_apps.md +++ b/doc/development/testing_guide/review_apps.md @@ -257,7 +257,7 @@ find a way to limit it to only us.** ## Other resources -* [Review Apps integration for CE/EE (presentation)](https://docs.google.com/presentation/d/1QPLr6FO4LduROU8pQIPkX1yfGvD13GEJIBOenqoKxR8/edit?usp=sharing) +- [Review Apps integration for CE/EE (presentation)](https://docs.google.com/presentation/d/1QPLr6FO4LduROU8pQIPkX1yfGvD13GEJIBOenqoKxR8/edit?usp=sharing) [charts-1068]: https://gitlab.com/charts/gitlab/issues/1068 [gitlab-pipeline]: https://gitlab.com/gitlab-org/gitlab-ce/pipelines/44362587 -- cgit v1.2.1 From e108f0d9610d0e019791920bb5adbc0b493d7b74 Mon Sep 17 00:00:00 2001 From: Marcel Amirault Date: Thu, 4 Jul 2019 22:46:12 +0000 Subject: Small markdown tweaks to clear kramdown warnings Fixing square brackets, links, etc in: merge_requests.md vulnerabilities.md issues.md issues_statistics.md pipelines.md services.md group_milestones.md milestones.md runners.md changelog.md issue_workflow.md elasticsearch.md api_graphql_styleguide.md automatic_ce_ee_merge.md file_storage.md architecture.md database_debugging.md index.md index.md frontend_testing.md pry_debugging.md vue.md development_process.md --- doc/development/testing_guide/end_to_end/index.md | 2 +- doc/development/testing_guide/frontend_testing.md | 1 - doc/development/testing_guide/index.md | 2 -- 3 files changed, 1 insertion(+), 4 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md index 527cd350633..59eb3ecfd7e 100644 --- a/doc/development/testing_guide/end_to_end/index.md +++ b/doc/development/testing_guide/end_to_end/index.md @@ -79,7 +79,7 @@ subgraph gitlab-ce/ee pipeline end subgraph omnibus-gitlab pipeline - A2[`Trigger-docker` stage
`Trigger:gitlab-docker` job] -->|once done| B2 + A2[`Trigger-docker` stage
`Trigger:gitlab-docker` job] -->|once done| B2 end subgraph gitlab-qa pipeline diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index 28ebb6f0f64..98df0b5ea7c 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -560,7 +560,6 @@ end [vue-test]: https://docs.gitlab.com/ce/development/fe_guide/vue.html#testing-vue-components [rspec]: https://github.com/rspec/rspec-rails#feature-specs [capybara]: https://github.com/teamcapybara/capybara -[karma]: http://karma-runner.github.io/ [jasmine]: https://jasmine.github.io/ --- diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md index 93ee2a6371a..c4b18391cb2 100644 --- a/doc/development/testing_guide/index.md +++ b/doc/development/testing_guide/index.md @@ -80,8 +80,6 @@ Everything you should know about how to run end-to-end tests using [Return to Development documentation](../README.md) -[^1]: /ci/yaml/README.html#dependencies - [rails]: http://rubyonrails.org/ [RSpec]: https://github.com/rspec/rspec-rails#feature-specs [Capybara]: https://github.com/teamcapybara/capybara -- cgit v1.2.1 From 6feda1df3b53df3d2fb9f8ac076049fbf3ffc0f1 Mon Sep 17 00:00:00 2001 From: Marcel Amirault Date: Fri, 5 Jul 2019 16:25:58 +0000 Subject: Change absolute links to relative Change assorted absolute links to ce and ee docs to relative links --- doc/development/testing_guide/end_to_end/quick_start_guide.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md index 041bdf716b3..064fb0e31dd 100644 --- a/doc/development/testing_guide/end_to_end/quick_start_guide.md +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -394,15 +394,15 @@ end By defining the `api_get_path` method, we allow the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to get a single issue. -> This `GET` path can be found in the [public API documentation](https://docs.gitlab.com/ee/api/issues.html#single-issue). +> This `GET` path can be found in the [public API documentation](../../../api/issues.md#single-issue). By defining the `api_post_path` method, we allow the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to create a new issue in a specific project. -> This `POST` path can be found in the [public API documentation](https://docs.gitlab.com/ee/api/issues.html#new-issue). +> This `POST` path can be found in the [public API documentation](../../../api/issues.md#new-issue). By defining the `api_post_body` method, we allow the [`ApiFabricator.api_post`](https://gitlab.com/gitlab-org/gitlab-ee/blob/a9177ca1812bac57e2b2fa4560e1d5dd8ffac38b/qa/qa/resource/api_fabricator.rb#L68) method to know which data to send when making the `POST` request. -> Notice that we pass both `labels` and `title` attributes in the `api_post_body`, where `labels` receives an array of labels, and [`title` is required](https://docs.gitlab.com/ee/api/issues.html#new-issue). Also, notice that we keep them alphabetically organized. +> Notice that we pass both `labels` and `title` attributes in the `api_post_body`, where `labels` receives an array of labels, and [`title` is required](../../../api/issues.md#new-issue). Also, notice that we keep them alphabetically organized. **Label resource** @@ -441,7 +441,7 @@ By defining the `api_post_path` method, we allow for the [`ApiFabricator `](http By defining the `api_post_body` method, we we allow for the [`ApiFabricator.api_post`](https://gitlab.com/gitlab-org/gitlab-ee/blob/a9177ca1812bac57e2b2fa4560e1d5dd8ffac38b/qa/qa/resource/api_fabricator.rb#L68) method to know which data to send when making the `POST` request. -> Notice that we pass both `color` and `name` attributes in the `api_post_body` since [those are required](https://docs.gitlab.com/ee/api/labels.html#create-a-new-label). Also, notice that we keep them alphabetically organized. +> Notice that we pass both `color` and `name` attributes in the `api_post_body` since [those are required](../../../api/labels.md#create-a-new-label). Also, notice that we keep them alphabetically organized. ### 8. Page Objects -- cgit v1.2.1 From 4f93fa4b7d8322aa1f0b39066d65a83c9b10977e Mon Sep 17 00:00:00 2001 From: Evan Read Date: Mon, 8 Jul 2019 11:07:07 +0000 Subject: Add lint rule that headings must increment one level at a time Also fixes instances in Markdown that broke the rule. --- doc/development/testing_guide/ci.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/ci.md b/doc/development/testing_guide/ci.md index 7a7fca46534..87d48726268 100644 --- a/doc/development/testing_guide/ci.md +++ b/doc/development/testing_guide/ci.md @@ -1,6 +1,6 @@ # GitLab tests in the Continuous Integration (CI) context -### Test suite parallelization on the CI +## Test suite parallelization on the CI Our current CI parallelization setup is as follows: @@ -26,7 +26,7 @@ Our current CI parallelization setup is as follows: After that, the next pipeline will use the up-to-date `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file. -### Monitoring +## Monitoring The GitLab test suite is [monitored] for the `master` branch, and any branch that includes `rspec-profile` in their name. -- cgit v1.2.1 From 347a8799096d2ace4e1b4577f8f79518085ffa40 Mon Sep 17 00:00:00 2001 From: Marcel Amirault Date: Tue, 9 Jul 2019 00:20:40 +0000 Subject: Update redirected links to final destination Updating redirected links in the development, install, policy raketasks and university dirs. Should now link to final URL to avoid redirect hops --- doc/development/testing_guide/index.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md index c4b18391cb2..aadbea1a540 100644 --- a/doc/development/testing_guide/index.md +++ b/doc/development/testing_guide/index.md @@ -11,7 +11,7 @@ importance. ## Overview -GitLab is built on top of [Ruby on Rails][rails], and we're using [RSpec] for all +GitLab is built on top of [Ruby on Rails](https://rubyonrails.org/), and we're using [RSpec] for all the backend tests, with [Capybara] for end-to-end integration testing. On the frontend side, we're using [Karma] and [Jasmine] for JavaScript unit and integration testing. @@ -80,7 +80,6 @@ Everything you should know about how to run end-to-end tests using [Return to Development documentation](../README.md) -[rails]: http://rubyonrails.org/ [RSpec]: https://github.com/rspec/rspec-rails#feature-specs [Capybara]: https://github.com/teamcapybara/capybara [Karma]: http://karma-runner.github.io/ -- cgit v1.2.1 From 143872fa52dfc37396e2ede946f1fe7fa9a8d20c Mon Sep 17 00:00:00 2001 From: Marcel Amirault Date: Fri, 12 Jul 2019 08:15:38 +0000 Subject: Remove trailing whitespace in docs Remove unneeded trailing whitespace in lines in /development /ci /api and /administration docs. --- doc/development/testing_guide/end_to_end/index.md | 2 +- doc/development/testing_guide/end_to_end/style_guide.md | 12 ++++++------ doc/development/testing_guide/frontend_testing.md | 8 ++++---- 3 files changed, 11 insertions(+), 11 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md index 59eb3ecfd7e..2dc06ba10a5 100644 --- a/doc/development/testing_guide/end_to_end/index.md +++ b/doc/development/testing_guide/end_to_end/index.md @@ -148,7 +148,7 @@ Once you decided where to put [test environment orchestration scenarios] and the [GitLab QA orchestrator README][gitlab-qa-readme], and [the already existing instance-level scenarios][instance-level scenarios]. -Continued reading: +Continued reading: - [Quick Start Guide](quick_start_guide.md) - [Style Guide](style_guide.md) diff --git a/doc/development/testing_guide/end_to_end/style_guide.md b/doc/development/testing_guide/end_to_end/style_guide.md index 0272e1810f2..52a8116e01c 100644 --- a/doc/development/testing_guide/end_to_end/style_guide.md +++ b/doc/development/testing_guide/end_to_end/style_guide.md @@ -63,17 +63,17 @@ We follow a simple formula roughly based on hungarian notation. - `_checkbox` - `_radio` - `_content` - + *Note: This list is a work in progress. This list will eventually be the end-all enumeration of all available types. I.e., any element that does not end with something in this list is bad form.* - + #### Examples **Good** ```ruby view '...' do - element :edit_button + element :edit_button element :notes_tab element :squash_checkbox element :username_field @@ -84,15 +84,15 @@ end **Bad** ```ruby -view '...' do +view '...' do # `_confirmation` should be `_field`. what sort of confirmation? a checkbox confirmation? no real way to disambiguate. # an appropriate replacement would be `element :password_confirmation_field` element :password_confirmation - # `clone_options` is too vague. If it's a dropdown menu, it should be `clone_dropdown`. + # `clone_options` is too vague. If it's a dropdown menu, it should be `clone_dropdown`. # If it's a checkbox, it should be `clone_checkbox` element :clone_options - + # how is this url being displayed? is it a textbox? a simple span? element :ssh_clone_url end diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index 98df0b5ea7c..bb44cc595e9 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -501,17 +501,17 @@ The following are examples of tests that work for both Karma and Jest: it('makes a request', () => { const responseBody = getJSONFixture('some/fixture.json'); // loads spec/javascripts/fixtures/some/fixture.json axiosMock.onGet(endpoint).reply(200, responseBody); - + myButton.click(); - + // ... }); it('uses some HTML element', () => { loadFixtures('some/page.html'); // loads spec/javascripts/fixtures/some/page.html and adds it to the DOM - + const element = document.getElementById('#my-id'); - + // ... }); ``` -- cgit v1.2.1 From c53d9e727858f80f8bd17e1831e00ba69b2b34fb Mon Sep 17 00:00:00 2001 From: Walmyr Lima Date: Fri, 12 Jul 2019 13:26:28 +0200 Subject: Fix link on end-to-end quick start guide doc --- doc/development/testing_guide/end_to_end/quick_start_guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md index 064fb0e31dd..efcfd44bc22 100644 --- a/doc/development/testing_guide/end_to_end/quick_start_guide.md +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -222,7 +222,7 @@ As the pre-conditions for our test suite, the things that needs to happen before - A project being created with an issue and labels already set; - The issue page being opened with only one scoped label applied to it. -> When running end-to-end tests as part of the GitLab's continuous integration process [a license is already set as an environment variable](https://gitlab.com/gitlab-org/gitlab-ee/blob/1a60d926740db10e3b5724713285780a4f470531/qa/qa/ee/strategy.rb#L20). For running tests locally you can set up such license by following the document [what tests can be run?](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/what_tests_can_be_run.md#supported-remote-grid-environment-variables), based on the [supported GitLab environment variables](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/what_tests_can_be_run.md#supported-gitlab-environment-variables). +> When running end-to-end tests as part of the GitLab's continuous integration process [a license is already set as an environment variable](https://gitlab.com/gitlab-org/gitlab-ee/blob/1a60d926740db10e3b5724713285780a4f470531/qa/qa/ee/strategy.rb#L20). For running tests locally you can set up such license by following the document [what tests can be run?](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/what_tests_can_be_run.md), based on the [supported GitLab environment variables](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/what_tests_can_be_run.md#supported-gitlab-environment-variables). #### Implementation -- cgit v1.2.1 From 4121c636763e4bcbca7622e9f0e341d5e0e5a3e1 Mon Sep 17 00:00:00 2001 From: Marcel Amirault Date: Mon, 15 Jul 2019 01:04:35 +0000 Subject: Remove extra blank lines from docs Removing extra blank lines in docs that serve no purpose, as well as some --- lines that also are not needed. --- doc/development/testing_guide/index.md | 18 ------------------ 1 file changed, 18 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md index aadbea1a540..96e8c30a679 100644 --- a/doc/development/testing_guide/index.md +++ b/doc/development/testing_guide/index.md @@ -22,62 +22,44 @@ automated testing means, and what are its principles: - [Five Factor Testing](https://www.devmynd.com/blog/five-factor-testing): Why do we need tests? - [Principles of Automated Testing](http://www.lihaoyi.com/post/PrinciplesofAutomatedTesting.html): Levels of testing. Prioritize tests. Cost of tests. ---- - ## [Testing levels](testing_levels.md) Learn about the different testing levels, and how to decide at what level your changes should be tested. ---- - ## [Testing best practices](best_practices.md) Everything you should know about how to write good tests: Test Design, RSpec, FactoryBot, system tests, parameterized tests etc. ---- - ## [Frontend testing standards and style guidelines](frontend_testing.md) Everything you should know about how to write good Frontend tests: Karma, testing promises, stubbing etc. ---- - ## [Flaky tests](flaky_tests.md) What are flaky tests, the different kind of flaky tests we encountered, and what we do about them. ---- - ## [GitLab tests in the Continuous Integration (CI) context](ci.md) How GitLab test suite is run in the CI context: setup, caches, artifacts, parallelization, monitoring. ---- - ## [Review apps](review_apps.md) How review apps are set up for GitLab CE/EE and how to use them. ---- - ## [Testing Rake tasks](testing_rake_tasks.md) Everything you should know about how to test Rake tasks. ---- - ## [End-to-end tests](end_to_end/index.md) Everything you should know about how to run end-to-end tests using [GitLab QA][gitlab-qa] testing framework. ---- - [Return to Development documentation](../README.md) [RSpec]: https://github.com/rspec/rspec-rails#feature-specs -- cgit v1.2.1 From 18b3f530fc58ff0278872d11d4f44324765b6347 Mon Sep 17 00:00:00 2001 From: ddavison Date: Mon, 15 Jul 2019 15:29:20 -0700 Subject: Add an appropriate solution in documentation A "bad" example was given, however no solution was provided for fixing the bad example --- doc/development/testing_guide/end_to_end/style_guide.md | 1 + 1 file changed, 1 insertion(+) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/style_guide.md b/doc/development/testing_guide/end_to_end/style_guide.md index 52a8116e01c..6a888142575 100644 --- a/doc/development/testing_guide/end_to_end/style_guide.md +++ b/doc/development/testing_guide/end_to_end/style_guide.md @@ -94,6 +94,7 @@ view '...' do element :clone_options # how is this url being displayed? is it a textbox? a simple span? + # If it is content on the page, it should be `ssh_clone_url_content` element :ssh_clone_url end ``` -- cgit v1.2.1 From e424fcedcc5cbe9cd96b74b057f243161db6e2e9 Mon Sep 17 00:00:00 2001 From: Martin Hanzel Date: Wed, 17 Jul 2019 17:47:19 +0000 Subject: Add docs about auto-injected Jest mocks --- doc/development/testing_guide/frontend_testing.md | 28 +++++++++++++++++++++++ 1 file changed, 28 insertions(+) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index bb44cc595e9..c909745b1ab 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -79,6 +79,34 @@ describe('Component', () => { Remember that the performance of each test depends on the environment. +### Manual module mocks +Jest supports [manual module mocks](https://jestjs.io/docs/en/manual-mocks) by placing a mock in a `__mocks__/` directory next to the source module. **Don't do this.** We want to keep all of our test-related code in one place (the `spec/` folder), and the logic that Jest uses to apply mocks from `__mocks__/` is rather inconsistent. + +Instead, our test runner detects manual mocks from `spec/frontend/mocks/`. Any mock placed here is automatically picked up and injected whenever you import its source module. + +- Files in `spec/frontend/mocks/ce` will mock the corresponding CE module from `app/assets/javascripts`, mirroring the source module's path. + - Example: `spec/frontend/mocks/ce/lib/utils/axios_utils` will mock the module `~/lib/utils/axios_utils`. +- Files in `spec/frontend/mocks/node` will mock NPM packages of the same name or path. +- We don't support mocking EE modules yet. + +If a mock is found for which a source module doesn't exist, the test suite will fail. 'Virtual' mocks, or mocks that don't have a 1-to-1 association with a source module, are not supported yet. + +#### Writing a mock +Create a JS module in the appropriate place in `spec/frontend/mocks/`. That's it. It will automatically mock its source package in all tests. + +Make sure that your mock's export has the same format as the mocked module. So, if you're mocking a CommonJS module, you'll need to use `module.exports` instead of the ES6 `export`. + +It might be useful for a mock to expose a property that indicates if the mock was loaded. This way, tests can assert the presence of a mock without calling any logic and causing side-effects. The `~/lib/utils/axios_utils` module mock has such a property, `isMock`, that is `true` in the mock and undefined in the original class. Jest's mock functions also have a `mock` property that you can test. + +#### Bypassing mocks +If you ever need to import the original module in your tests, use [`jest.requireActual()`](https://jestjs.io/docs/en/jest-object#jestrequireactualmodulename) (or `jest.requireActual().default` for the default export). The `jest.mock()` and `jest.unmock()` won't have an effect on modules that have a manual mock, because mocks are imported and cached before any tests are run. + +#### Keep mocks light +Global mocks introduce magic and can affect how modules are imported in your tests. Try to keep them as light as possible and dependency-free. A global mock should be useful for any unit test. For example, the `axios_utils` and `jquery` module mocks throw an error when an HTTP request is attempted, since this is useful behaviour in >99% of tests. + +When in doubt, construct mocks in your test file using [`jest.mock()`](https://jestjs.io/docs/en/jest-object#jestmockmodulename-factory-options), [`jest.spyOn()`](https://jestjs.io/docs/en/jest-object#jestspyonobject-methodname), etc. + + ## Karma test suite GitLab uses the [Karma][karma] test runner with [Jasmine] as its test -- cgit v1.2.1 From a7e16ee20893bbc7f31fb0eb3cdd3018ace9acda Mon Sep 17 00:00:00 2001 From: Dan Davison Date: Wed, 17 Jul 2019 19:30:00 +0000 Subject: Add documentation surrounding [data-qa-selector] Documentation was lacking for the [data-qa-selector] method of defining methods vs .qa-selector method. --- .../testing_guide/end_to_end/page_objects.md | 38 +++++++++++++++++----- .../testing_guide/end_to_end/quick_start_guide.md | 33 +++++++++---------- 2 files changed, 45 insertions(+), 26 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/page_objects.md b/doc/development/testing_guide/end_to_end/page_objects.md index 05cb03eb4bd..29ad49403fe 100644 --- a/doc/development/testing_guide/end_to_end/page_objects.md +++ b/doc/development/testing_guide/end_to_end/page_objects.md @@ -92,20 +92,25 @@ end The `view` DSL method will correspond to the rails View, partial, or vue component that renders the elements. The `element` DSL method in turn declares an element for which a corresponding -`qa-element-name-dasherized` CSS class will need to be added to the view file. +`data-qa-selector=element_name_snaked` data attribute will need to be added to the view file. You can also define a value (String or Regexp) to match to the actual view code but **this is deprecated** in favor of the above method for two reasons: - Consistency: there is only one way to define an element -- Separation of concerns: QA uses dedicated CSS classes instead of reusing code +- Separation of concerns: QA uses dedicated `data-qa-*` attributes instead of reusing code or classes used by other components (e.g. `js-*` classes etc.) ```ruby view 'app/views/my/view.html.haml' do - # Implicitly require `.qa-logout-button` CSS class to be present in the view + + ### Good ### + + # Implicitly require the CSS selector `[data-qa-selector="logout_button"]` to be present in the view element :logout_button + ### Bad ### + ## This is deprecated and forbidden by the `QA/ElementWithPattern` RuboCop cop. # Require `f.submit "Sign in"` to be present in `my/view.html.haml element :my_button, 'f.submit "Sign in"' # rubocop:disable QA/ElementWithPattern @@ -129,24 +134,39 @@ view 'app/views/my/view.html.haml' do end ``` -To add these elements to the view, you must change the rails View, partial, or vue component by adding a `qa-element-descriptor` class +To add these elements to the view, you must change the rails View, partial, or vue component by adding a `data-qa-selector` attribute for each element defined. -In our case, `qa-login-field`, `qa-password-field` and `qa-sign-in-button` +In our case, `data-qa-selector="login_field"`, `data-qa-selector="password_field"` and `data-qa-selector="sign_in_button"` **app/views/my/view.html.haml** ```haml -= f.text_field :login, class: "form-control top qa-login-field", autofocus: "autofocus", autocapitalize: "off", autocorrect: "off", required: true, title: "This field is required." -= f.password_field :password, class: "form-control bottom qa-password-field", required: true, title: "This field is required." -= f.submit "Sign in", class: "btn btn-success qa-sign-in-button" += f.text_field :login, class: "form-control top", autofocus: "autofocus", autocapitalize: "off", autocorrect: "off", required: true, title: "This field is required.", data: { qa_selector: 'login_field' } += f.password_field :password, class: "form-control bottom", required: true, title: "This field is required.", data: { qa_selector: 'password_field' } += f.submit "Sign in", class: "btn btn-success", data: { qa_selector: 'sign_in_button' } ``` Things to note: -- The CSS class must be `kebab-cased` (separated with hyphens "`-`") +- The name of the element and the qa_selector must match and be snake_cased - If the element appears on the page unconditionally, add `required: true` to the element. See [Dynamic element validation](dynamic_element_validation.md) +- You may see `.qa-selector` classes in existing Page Objects. We should prefer the [`data-qa-selector`](#data-qa-selector-vs-qa-selector) + method of definition over the `.qa-selector` CSS class + + +### `data-qa-selector` vs `.qa-selector` + +> Introduced in GitLab 12.1 + +There are two supported methods of defining elements within a view. + +1. `data-qa-selector` attribute +1. `.qa-selector` class + +Any existing `.qa-selector` class should be considered deprecated +and we should prefer the `data-qa-selector` method of definition. ## Running the test locally diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md index efcfd44bc22..3bbf8feab39 100644 --- a/doc/development/testing_guide/end_to_end/quick_start_guide.md +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -101,7 +101,7 @@ it 'replaces an existing label if it has the same key' do page.find('#content-body').click page.refresh - labels_block = page.find('.qa-labels-block') + labels_block = page.find(%q([data-qa-selector="labels_block"])) expect(labels_block).to have_content('animal::dolphin') expect(labels_block).not_to have_content('animal::fox') @@ -130,7 +130,7 @@ it 'keeps both scoped labels when adding a label with a different key' do page.find('#content-body').click page.refresh - labels_block = page.find('.qa-labels-block') + labels_block = page.find(%q([data-qa-selector="labels_block"])) expect(labels_block).to have_content('animal::fox') expect(labels_block).to have_content('plant::orchid') @@ -139,7 +139,7 @@ it 'keeps both scoped labels when adding a label with a different key' do end ``` -> Note that elements are always located using CSS selectors, and a good practice is to add test-specific selectors (this is called adding testability to the application and we will talk more about it later.) For example, the `labels_block` element uses the selector `.qa-labels-block`, which was added specifically for testing purposes. +> Note that elements are always located using CSS selectors, and a good practice is to add test-specific selectors (this is called "testability"). For example, the `labels_block` element uses the CSS selector [`data-qa-selector="labels_block"`](page_objects.md#data-qa-selector-vs-qa-selector), which was added specifically for testing purposes. Below are the steps that the test covers: @@ -168,7 +168,7 @@ end it 'replaces an existing label if it has the same key' do select_label_and_refresh @new_label_same_scope - labels_block = page.find('.qa-labels-block') + labels_block = page.find(%q([data-qa-selector="labels_block"])) expect(labels_block).to have_content(@new_label_same_scope) expect(labels_block).not_to have_content(@initial_label) @@ -179,7 +179,7 @@ end it 'keeps both scoped label when adding a label with a different key' do select_label_and_refresh @new_label_different_scope - labels_block = page.find('.qa-labels-block') + labels_block = page.find(%q([data-qa-selector="labels_block"])) expect(labels_blocks).to have_content(@new_label_different_scope) expect(labels_blocks).to have_content(@initial_label) @@ -305,7 +305,7 @@ module QA it 'correctly applies scoped labels depending on if they are from the same or a different scope' do select_labels_and_refresh [@new_label_same_scope, @new_label_different_scope] - labels_block = page.all('.qa-labels-block') + labels_block = page.all(%q([data-qa-selector="labels_block"])) expect(labels_block).to have_content(@new_label_same_scope) expect(labels_block).to have_content(@new_label_different_scope) @@ -552,37 +552,36 @@ The `text_of_labels_block` method is a simple method that returns the `:labels_b #### Updates in the view (*.html.haml) and `dropdowns_helper.rb` files -Now let's change the view and the `dropdowns_helper` files to add the selectors that relate to the Page Object. +Now let's change the view and the `dropdowns_helper` files to add the selectors that relate to the [Page Objects]. -In the [app/views/shared/issuable/_sidebar.html.haml](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/app/views/shared/issuable/_sidebar.html.haml) file, on [line 105 ](https://gitlab.com/gitlab-org/gitlab-ee/blob/84043fa72ca7f83ae9cde48ad670e6d5d16501a3/app/views/shared/issuable/_sidebar.html.haml#L105), add an extra class `qa-edit-link-labels`. +In [`app/views/shared/issuable/_sidebar.html.haml:105`](https://gitlab.com/gitlab-org/gitlab-ee/blob/7ca12defc7a965987b162a6ebef302f95dc8867f/app/views/shared/issuable/_sidebar.html.haml#L105), add a `data: { qa_selector: 'edit_link_labels' }` data attribute. The code should look like this: ```haml -= link_to _('Edit'), '#', class: 'js-sidebar-dropdown-toggle edit-link float-right qa-edit-link-labels' += link_to _('Edit'), '#', class: 'js-sidebar-dropdown-toggle edit-link float-right', data: { qa_selector: 'edit_link_labels' } ``` -In the same file, on [line 121](https://gitlab.com/gitlab-org/gitlab-ee/blob/84043fa72ca7f83ae9cde48ad670e6d5d16501a3/app/views/shared/issuable/_sidebar.html.haml#L121), add an extra class `.qa-dropdown-menu-labels`. +In the same file, on [line 121](https://gitlab.com/gitlab-org/gitlab-ee/blob/7ca12defc7a965987b162a6ebef302f95dc8867f/app/views/shared/issuable/_sidebar.html.haml#L121), add a `data: { qa_selector: 'dropdown_menu_labels' }` data attribute. The code should look like this: ```haml -.dropdown-menu.dropdown-select.dropdown-menu-paging.dropdown-menu-labels.dropdown-menu-selectable.qa-dropdown-menu-labels +.dropdown-menu.dropdown-select.dropdown-menu-paging.dropdown-menu-labels.dropdown-menu-selectable.dropdown-extended-height{ data: { qa_selector: 'dropdown_menu_labels' } } ``` -In the [`dropdowns_helper.rb`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/app/helpers/dropdowns_helper.rb) file, on [line 94](https://gitlab.com/gitlab-org/gitlab-ee/blob/99e51a374f2c20bee0989cac802e4b5621f72714/app/helpers/dropdowns_helper.rb#L94), add an extra class `qa-dropdown-input-field`. +In [`app/helpers/dropdowns_helper.rb:94`](https://gitlab.com/gitlab-org/gitlab-ee/blob/7ca12defc7a965987b162a6ebef302f95dc8867f/app/helpers/dropdowns_helper.rb#L94), add a `data: { qa_selector: 'dropdown_input_field' }` data attribute. The code should look like this: ```ruby -filter_output = search_field_tag search_id, nil, class: "dropdown-input-field qa-dropdown-input-field", placeholder: placeholder, autocomplete: 'off' +filter_output = search_field_tag search_id, nil, class: "dropdown-input-field", placeholder: placeholder, autocomplete: 'off', data: { qa_selector: 'dropdown_input_field' } ``` -> Classes starting with `qa-` are used for testing purposes only, and by defining such classes in the elements we add **testability** in the application. +> `data-qa-*` data attributes and CSS classes starting with `qa-` are used solely for the purpose of QA and testing. +> By defining these, we add **testability** to the application. -> When defining a class like `qa-labels-block`, it is transformed into `:labels_block` for usage in the Page Objects. So, `qa-edit-link-labels` is transformed into `:edit_link_labels`, `qa-dropdown-menu-labels` is transformed into `:dropdown_menu_labels`, and `qa-dropdown-input-field` is transformed into `:dropdown_input_field`. Also, we use a [sanity test](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/qa/page#how-did-we-solve-fragile-tests-problem) to check that defined elements have their respective `qa-` selectors in the specified views. - -> We did not define the `qa-labels-block` class in the `app/views/shared/issuable/_sidebar.html.haml` file because it was already there to be used. +> When defining a data attribute like: `qa_selector: 'labels_block'`, it should match the element definition: `element :labels_block`. We use a [sanity test](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/qa/page#how-did-we-solve-fragile-tests-problem) to check that defined elements have their respective selectors in the specified views. #### Updates in the `QA::Page::Base` class -- cgit v1.2.1 From 746f54787799ee5ea8595a8730d363bfd250ffab Mon Sep 17 00:00:00 2001 From: Marcel Amirault Date: Thu, 18 Jul 2019 01:15:58 +0000 Subject: Fix unordered list spacing Correct the spacing of unordered markdown lists in docs, to maintain standards of documentation. --- doc/development/testing_guide/review_apps.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md index ae40d628717..96761622cfe 100644 --- a/doc/development/testing_guide/review_apps.md +++ b/doc/development/testing_guide/review_apps.md @@ -137,8 +137,8 @@ secure note named **gitlab-{ce,ee} Review App's root password**. ### Run a Rails console -1. [Filter Workloads by your Review App slug](https://console.cloud.google.com/kubernetes/workload?project=gitlab-review-apps) - , e.g. `review-qa-raise-e-12chm0`. +1. [Filter Workloads by your Review App slug](https://console.cloud.google.com/kubernetes/workload?project=gitlab-review-apps), + e.g. `review-qa-raise-e-12chm0`. 1. Find and open the `task-runner` Deployment, e.g. `review-qa-raise-e-12chm0-task-runner`. 1. Click on the Pod in the "Managed pods" section, e.g. `review-qa-raise-e-12chm0-task-runner-d5455cc8-2lsvz`. 1. Click on the `KUBECTL` dropdown, then `Exec` -> `task-runner`. @@ -196,7 +196,7 @@ For the record, the debugging steps to find out this issue were: 1. `kubectl describe pod ` & confirm exact error message 1. Web search for exact error message, following rabbit hole to [a relevant kubernetes bug report](https://github.com/kubernetes/kubernetes/issues/57345) 1. Access the node over SSH via the GCP console (**Computer Engine > VM - instances** then click the "SSH" button for the node where the `dns-gitlab-review-app-external-dns` pod runs) + instances** then click the "SSH" button for the node where the `dns-gitlab-review-app-external-dns` pod runs) 1. In the node: `systemctl --version` => systemd 232 1. Gather some more information: - `mount | grep kube | wc -l` => e.g. 290 @@ -211,7 +211,7 @@ For the record, the debugging steps to find out this issue were: To resolve the problem, we needed to (forcibly) drain some nodes: 1. Try a normal drain on the node where the `dns-gitlab-review-app-external-dns` - pod runs so that Kubernetes automatically move it to another node: `kubectl drain NODE_NAME` + pod runs so that Kubernetes automatically move it to another node: `kubectl drain NODE_NAME` 1. If that doesn't work, you can also perform a forcible "drain" the node by removing all pods: `kubectl delete pods --field-selector=spec.nodeName=NODE_NAME` 1. In the node: - Perform `systemctl daemon-reload` to remove the dead/inactive units -- cgit v1.2.1 From 0604390a111d6d3410d93a69cbc321a934b21f41 Mon Sep 17 00:00:00 2001 From: Dan Davison Date: Thu, 18 Jul 2019 19:21:02 +0000 Subject: Fix QA documentation to refer to perform method The documentation referred to the .act method This MR changes this to refer to the preferred .perform method --- doc/development/testing_guide/end_to_end/page_objects.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/page_objects.md b/doc/development/testing_guide/end_to_end/page_objects.md index 29ad49403fe..52957d1a1ab 100644 --- a/doc/development/testing_guide/end_to_end/page_objects.md +++ b/doc/development/testing_guide/end_to_end/page_objects.md @@ -27,7 +27,7 @@ When someone later changes `t.text_field :login` in the view associated with this page to `t.text_field :username` it will generate a different field identifier, what would effectively break all tests. -Because we are using `Page::Main::Login.act { sign_in_using_credentials }` +Because we are using `Page::Main::Login.perform(&:sign_in_using_credentials)` everywhere, when we want to sign into GitLab, the page object is the single source of truth, and we will need to update `fill_in :user_login` to `fill_in :user_username` only in a one place. -- cgit v1.2.1 From 7da80b2d36adc30964423042e956ef880a2650f9 Mon Sep 17 00:00:00 2001 From: Marcel Amirault Date: Fri, 19 Jul 2019 02:20:32 +0000 Subject: Update numbered lists for docs standards Ensure that all numbered lists use only 1. and no other numbers. Also ensure that numbered lists use proper spacing. --- .../testing_guide/end_to_end/quick_start_guide.md | 24 +++++++++++----------- 1 file changed, 12 insertions(+), 12 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md index 3bbf8feab39..14a169dcc1d 100644 --- a/doc/development/testing_guide/end_to_end/quick_start_guide.md +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -116,9 +116,9 @@ end Below are the steps that the test covers: 1. The test finds the 'Edit' link for the labels and clicks on it. -2. Then it fills in the 'Assign labels' input field with the value 'animal::dolphin' and press enters. -3. Then it clicks in the content body to apply the label and refreshes the page. -4. Finally, the expectations check that the previous scoped label was removed and that the new one was added. +1. Then it fills in the 'Assign labels' input field with the value 'animal::dolphin' and press enters. +1. Then it clicks in the content body to apply the label and refreshes the page. +1. Finally, the expectations check that the previous scoped label was removed and that the new one was added. Let's now see how the second test case would look. @@ -144,9 +144,9 @@ end Below are the steps that the test covers: 1. The test finds the 'Edit' link for the labels and clicks on it. -2. Then it fills in the 'Assign labels' input field with the value 'plant::orchid' and press enters. -3. Then it clicks in the content body to apply the label and refreshes the page. -4. Finally, the expectations check that both scoped labels are present. +1. Then it fills in the 'Assign labels' input field with the value 'plant::orchid' and press enters. +1. Then it clicks in the content body to apply the label and refreshes the page. +1. Finally, the expectations check that both scoped labels are present. > Similar to the previous test, this one is also very straightforward, but there is some code duplication. Let's address it. @@ -290,7 +290,7 @@ As already mentioned in the [best practices](best_practices.md) document, end-to Some improvements that we could make in our test suite to optimize its time to run are: 1. Having a single test case (an `it` block) that exercises both scenarios to avoid "wasting" time in the tests' pre-conditions, instead of having two different test cases. -2. Making the selection of labels more performant by allowing for the selection of more than one label in the same reusable method. +1. Making the selection of labels more performant by allowing for the selection of more than one label in the same reusable method. Let's look at a suggestion that addresses the above points, one by one: @@ -332,8 +332,8 @@ To address point 1, we changed the test implementation from two `it` blocks into > Notice that the implementation of the new and unique `it` block had to change a little bit. Below we describe in details what it does. 1. It selects two scoped labels simultaneously, one from the same scope of the one already applied in the issue during the setup phase (in the `before` block), and another one from a different scope. -2. It asserts that the correct labels are visible in the `labels_block`, and that the labels were correctly added and removed; -3. Finally, the `select_label_and_refresh` method is changed to `select_labels_and_refresh`, which accepts an array of labels instead of a single label, and it iterates on them for faster label selection (this is what is used in step 1 explained above.) +1. It asserts that the correct labels are visible in the `labels_block`, and that the labels were correctly added and removed; +1. Finally, the `select_label_and_refresh` method is changed to `select_labels_and_refresh`, which accepts an array of labels instead of a single label, and it iterates on them for faster label selection (this is what is used in step 1 explained above.) ### 7. Resources @@ -542,9 +542,9 @@ end Notice that we have not only moved the `select_labels_and_refresh` method, but we have also changed its implementation to: 1. Click the `:edit_link_labels` element previously defined, instead of using `find('.block.labels .edit-link').click` -2. Use `within_element(:dropdown_menu_labels, text: label)`, and inside of it, we call `send_keys_to_element(:dropdown_input_field, [label, :enter])`, which is a method that we will implement in the `QA::Page::Base` class to replace `find('.dropdown-menu-labels .dropdown-input-field').send_keys [label, :enter]` -3. Use `click_body` after iterating on each label, instead of using `find('#content-body').click` -4. Iterate on every label again, and then we use `has_element?(:labels_block, text: label)` after clicking the page body (which applies the labels), and before refreshing the page, to avoid test flakiness due to refreshing too fast. +1. Use `within_element(:dropdown_menu_labels, text: label)`, and inside of it, we call `send_keys_to_element(:dropdown_input_field, [label, :enter])`, which is a method that we will implement in the `QA::Page::Base` class to replace `find('.dropdown-menu-labels .dropdown-input-field').send_keys [label, :enter]` +1. Use `click_body` after iterating on each label, instead of using `find('#content-body').click` +1. Iterate on every label again, and then we use `has_element?(:labels_block, text: label)` after clicking the page body (which applies the labels), and before refreshing the page, to avoid test flakiness due to refreshing too fast. ##### Details of `text_of_labels_block` -- cgit v1.2.1 From faa17d526371a34325705b8de9594e4e9df03298 Mon Sep 17 00:00:00 2001 From: Mike Greiling Date: Wed, 17 Jul 2019 17:17:37 -0500 Subject: Rename the karma:fixtures task Introduces frontend:fixtures rake task to replace karma:fixtures and update documentation and CI scripts to match --- doc/development/testing_guide/frontend_testing.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index c909745b1ab..f16037e9e0a 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -80,18 +80,20 @@ describe('Component', () => { Remember that the performance of each test depends on the environment. ### Manual module mocks + Jest supports [manual module mocks](https://jestjs.io/docs/en/manual-mocks) by placing a mock in a `__mocks__/` directory next to the source module. **Don't do this.** We want to keep all of our test-related code in one place (the `spec/` folder), and the logic that Jest uses to apply mocks from `__mocks__/` is rather inconsistent. Instead, our test runner detects manual mocks from `spec/frontend/mocks/`. Any mock placed here is automatically picked up and injected whenever you import its source module. - Files in `spec/frontend/mocks/ce` will mock the corresponding CE module from `app/assets/javascripts`, mirroring the source module's path. - - Example: `spec/frontend/mocks/ce/lib/utils/axios_utils` will mock the module `~/lib/utils/axios_utils`. + - Example: `spec/frontend/mocks/ce/lib/utils/axios_utils` will mock the module `~/lib/utils/axios_utils`. - Files in `spec/frontend/mocks/node` will mock NPM packages of the same name or path. - We don't support mocking EE modules yet. If a mock is found for which a source module doesn't exist, the test suite will fail. 'Virtual' mocks, or mocks that don't have a 1-to-1 association with a source module, are not supported yet. #### Writing a mock + Create a JS module in the appropriate place in `spec/frontend/mocks/`. That's it. It will automatically mock its source package in all tests. Make sure that your mock's export has the same format as the mocked module. So, if you're mocking a CommonJS module, you'll need to use `module.exports` instead of the ES6 `export`. @@ -99,14 +101,15 @@ Make sure that your mock's export has the same format as the mocked module. So, It might be useful for a mock to expose a property that indicates if the mock was loaded. This way, tests can assert the presence of a mock without calling any logic and causing side-effects. The `~/lib/utils/axios_utils` module mock has such a property, `isMock`, that is `true` in the mock and undefined in the original class. Jest's mock functions also have a `mock` property that you can test. #### Bypassing mocks + If you ever need to import the original module in your tests, use [`jest.requireActual()`](https://jestjs.io/docs/en/jest-object#jestrequireactualmodulename) (or `jest.requireActual().default` for the default export). The `jest.mock()` and `jest.unmock()` won't have an effect on modules that have a manual mock, because mocks are imported and cached before any tests are run. #### Keep mocks light + Global mocks introduce magic and can affect how modules are imported in your tests. Try to keep them as light as possible and dependency-free. A global mock should be useful for any unit test. For example, the `axios_utils` and `jquery` module mocks throw an error when an HTTP request is attempted, since this is useful behaviour in >99% of tests. When in doubt, construct mocks in your test file using [`jest.mock()`](https://jestjs.io/docs/en/jest-object#jestmockmodulename-factory-options), [`jest.spyOn()`](https://jestjs.io/docs/en/jest-object#jestspyonobject-methodname), etc. - ## Karma test suite GitLab uses the [Karma][karma] test runner with [Jasmine] as its test @@ -462,7 +465,7 @@ See this [section][vue-test]. For running the frontend tests, you need the following commands: -- `rake karma:fixtures` (re-)generates [fixtures](#frontend-test-fixtures). +- `rake frontend:fixtures` (re-)generates [fixtures](#frontend-test-fixtures). - `yarn test` executes the tests. As long as the fixtures don't change, `yarn test` is sufficient (and saves you some time). @@ -548,7 +551,7 @@ HTML and JSON fixtures are generated from backend views and controllers using RS For each fixture, the content of the `response` variable is stored in the output file. This variable gets automagically set if the test is marked as `type: :request` or `type: :controller`. -Fixtures are regenerated using the `bin/rake karma:fixtures` command but you can also generate them individually, +Fixtures are regenerated using the `bin/rake frontend:fixtures` command but you can also generate them individually, for example `bin/rspec spec/javascripts/fixtures/merge_requests.rb`. When creating a new fixture, it often makes sense to take a look at the corresponding tests for the endpoint in `(ee/)spec/controllers/` or `(ee/)spec/requests/`. -- cgit v1.2.1 From 97b8f85e03f0b6dcfc5d85b92323be979bd27b09 Mon Sep 17 00:00:00 2001 From: Mike Greiling Date: Thu, 18 Jul 2019 14:11:09 -0500 Subject: Move frontend fixtures into spec/frontend Moves every fixture source file from spec/javascripts to spec/frontend --- doc/development/testing_guide/frontend_testing.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index f16037e9e0a..ff28c2ea5e2 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -518,8 +518,8 @@ Information on setting up and running RSpec integration tests with Code that is added to HAML templates (in `app/views/`) or makes Ajax requests to the backend has tests that require HTML or JSON from the backend. Fixtures for these tests are located at: -- `spec/javascripts/fixtures/`, for running tests in CE. -- `ee/spec/javascripts/fixtures/`, for running tests in EE. +- `spec/frontend/fixtures/`, for running tests in CE. +- `ee/spec/frontend/fixtures/`, for running tests in EE. Fixture files in: @@ -530,7 +530,7 @@ The following are examples of tests that work for both Karma and Jest: ```javascript it('makes a request', () => { - const responseBody = getJSONFixture('some/fixture.json'); // loads spec/javascripts/fixtures/some/fixture.json + const responseBody = getJSONFixture('some/fixture.json'); // loads spec/frontend/fixtures/some/fixture.json axiosMock.onGet(endpoint).reply(200, responseBody); myButton.click(); @@ -539,7 +539,7 @@ it('makes a request', () => { }); it('uses some HTML element', () => { - loadFixtures('some/page.html'); // loads spec/javascripts/fixtures/some/page.html and adds it to the DOM + loadFixtures('some/page.html'); // loads spec/frontend/fixtures/some/page.html and adds it to the DOM const element = document.getElementById('#my-id'); @@ -547,12 +547,12 @@ it('uses some HTML element', () => { }); ``` -HTML and JSON fixtures are generated from backend views and controllers using RSpec (see `spec/javascripts/fixtures/*.rb`). +HTML and JSON fixtures are generated from backend views and controllers using RSpec (see `spec/frontend/fixtures/*.rb`). For each fixture, the content of the `response` variable is stored in the output file. This variable gets automagically set if the test is marked as `type: :request` or `type: :controller`. Fixtures are regenerated using the `bin/rake frontend:fixtures` command but you can also generate them individually, -for example `bin/rspec spec/javascripts/fixtures/merge_requests.rb`. +for example `bin/rspec spec/frontend/fixtures/merge_requests.rb`. When creating a new fixture, it often makes sense to take a look at the corresponding tests for the endpoint in `(ee/)spec/controllers/` or `(ee/)spec/requests/`. ## Gotchas -- cgit v1.2.1 From 0f7d8c04219a5631d48a2eff59d4a5daa881787f Mon Sep 17 00:00:00 2001 From: Marcel Amirault Date: Thu, 25 Jul 2019 22:49:13 +0000 Subject: Update links to relative Following the single docs codebase change, all internal links should be relative. Also cleans up one table --- doc/development/testing_guide/frontend_testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index ff28c2ea5e2..2985278cc92 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -588,7 +588,7 @@ end [jasmine-focus]: https://jasmine.github.io/2.5/focused_specs.html [karma]: http://karma-runner.github.io/ -[vue-test]: https://docs.gitlab.com/ce/development/fe_guide/vue.html#testing-vue-components +[vue-test]: ../fe_guide/vue.md#testing-vue-components [rspec]: https://github.com/rspec/rspec-rails#feature-specs [capybara]: https://github.com/teamcapybara/capybara [jasmine]: https://jasmine.github.io/ -- cgit v1.2.1 From 25cfdd69d3fbdd44d4fc2588ea2c0413d3a089df Mon Sep 17 00:00:00 2001 From: Walmyr Lima Date: Wed, 31 Jul 2019 15:27:18 +0200 Subject: Update sections titles with right heading --- doc/development/testing_guide/end_to_end/style_guide.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/style_guide.md b/doc/development/testing_guide/end_to_end/style_guide.md index 6a888142575..c9c375d4320 100644 --- a/doc/development/testing_guide/end_to_end/style_guide.md +++ b/doc/development/testing_guide/end_to_end/style_guide.md @@ -45,7 +45,7 @@ Notice that in the above example, before clicking the `:operations_environments_ > We can create these methods as helpers to abstract multi-step navigation. -### Element naming convention +## Element naming convention When adding new elements to a page, it's important that we have a uniform element naming convention. @@ -67,7 +67,7 @@ We follow a simple formula roughly based on hungarian notation. *Note: This list is a work in progress. This list will eventually be the end-all enumeration of all available types. I.e., any element that does not end with something in this list is bad form.* -#### Examples +### Examples **Good** -- cgit v1.2.1 From f26bb243ccf4e05f3f231d3ed62eb862dc643f0c Mon Sep 17 00:00:00 2001 From: Walmyr Lima Date: Wed, 31 Jul 2019 15:29:06 +0200 Subject: Update e2e style guide with new guideline --- .../testing_guide/end_to_end/style_guide.md | 44 ++++++++++++++++++++++ 1 file changed, 44 insertions(+) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/style_guide.md b/doc/development/testing_guide/end_to_end/style_guide.md index c9c375d4320..97560e616a1 100644 --- a/doc/development/testing_guide/end_to_end/style_guide.md +++ b/doc/development/testing_guide/end_to_end/style_guide.md @@ -98,3 +98,47 @@ view '...' do element :ssh_clone_url end ``` + +## Block argument naming + +To have a standard on how we call pages when using the `.perform` method, we use the name of page object being called, all lowercased, and separated by underscore, if needed (see good and bad examples below.) This also applies to resources. We chose not to simply use `page` because that would shadow the Capybara DSL, potentially leading to confusion and bugs. + +### Examples + +**Good** + +```ruby +# qa/specs/features/browser_ui/1_manage/project/add_project_member_spec.rb + +Page::Project::Settings::Members.perform do |members| + members.do_something +end +``` + +```ruby +# qa/specs/features/ee/browser_ui/3_create/merge_request/add_batch_comments_in_merge_request_spec.rb + +Resource::MergeRequest.fabricate! do |merge_request| + merge_request.do_something_else +end +``` + +**Bad** + +```ruby +# qa/specs/features/browser_ui/1_manage/project/add_project_member_spec.rb + +Page::Project::Settings::Members.perform do |project_settings_members_page| + project_settings_members_page.do_something +end +``` + +```ruby +# qa/specs/features/ee/browser_ui/3_create/merge_request/add_batch_comments_in_merge_request_spec.rb + +Resource::MergeRequest.fabricate! do |merge_request_page| + merge_request_page.do_something_else +end +``` + +> Besides the advantage of having a standard in place, by following this standard we also write shorter lines of code. \ No newline at end of file -- cgit v1.2.1 From aaefcb9c7e7e447a4434915ab3a84664a7f0dd4f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?R=C3=A9my=20Coutable?= Date: Thu, 1 Aug 2019 17:52:19 +0200 Subject: Document the current state of the Review App clusters MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Rémy Coutable --- doc/development/testing_guide/review_apps.md | 31 ++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 9 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md index 96761622cfe..d07a7045390 100644 --- a/doc/development/testing_guide/review_apps.md +++ b/doc/development/testing_guide/review_apps.md @@ -115,6 +115,28 @@ On every [pipeline][gitlab-pipeline] in the `qa` stage, the browser performance testing using a [Sitespeed.io Container](../../user/project/merge_requests/browser_performance_testing.md). +## Cluster configuration + +### Node pools + +Both `review-apps-ce` and `review-apps-ee` clusters are currently set up with +two node pools: + +- a node pool of non-preemptible `n1-standard-2` (2 vCPU, 7.5 GB memory) nodes + dedicated to the `tiller` deployment (see below) with a single node. +- a node pool of preemptible `n1-standard-2` (2 vCPU, 7.5 GB memory) nodes, + with a minimum of 1 node and a maximum of 250 nodes. + +### Helm/Tiller + +The `tiller` deployment (the Helm server) is deployed to a dedicated node pool +that has the `app=helm` label and a specific +[taint](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) +to prevent other pods from being scheduled on this node pool. + +This is to ensure Tiller isn't affected by "noisy" neighbors that could put +their node under pressure. + ## How to: ### Log into my Review App @@ -241,15 +263,6 @@ thousands of unused Docker images.** CNG-mirror project to store these Docker images so that we can just wipe out the registry at some point, and use a new fresh, empty one. -**How big are the Kubernetes clusters (`review-apps-ce` and `review-apps-ee`)?** - - > The clusters are currently set up with a single pool of preemptible nodes, - with a minimum of 1 node and a maximum of 500 nodes. - -**What are the machine running on the cluster?** - - > We're currently using `n1-standard-1` (1 vCPU, 3.75 GB memory) machines. - **How do we secure this from abuse? Apps are open to the world so we need to find a way to limit it to only us.** -- cgit v1.2.1 From a3e18e7c97b2a5d1f5695c71ac55cc91931c992e Mon Sep 17 00:00:00 2001 From: Stan Hu Date: Tue, 30 Jul 2019 23:22:22 -0700 Subject: Use Mermaid Markdown for testing guide docs * Remove PNGs in favor of dynamically rendered graphs * Eliminate styling because this is no longer supported * Change
to
because the latter is only supported * Add quotes around subgraph titles --- doc/development/testing_guide/end_to_end/index.md | 24 +++++++++------------ .../img/qa_on_merge_requests_cicd_architecture.png | Bin 64862 -> 0 bytes .../img/review_apps_cicd_architecture.png | Bin 136431 -> 0 bytes doc/development/testing_guide/review_apps.md | 23 ++++++++------------ 4 files changed, 19 insertions(+), 28 deletions(-) delete mode 100644 doc/development/testing_guide/img/qa_on_merge_requests_cicd_architecture.png delete mode 100644 doc/development/testing_guide/img/review_apps_cicd_architecture.png (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md index 2dc06ba10a5..882e2230636 100644 --- a/doc/development/testing_guide/end_to_end/index.md +++ b/doc/development/testing_guide/end_to_end/index.md @@ -65,28 +65,24 @@ Below you can read more about how to use it and how does it work. Currently, we are using _multi-project pipeline_-like approach to run QA pipelines. -![QA on merge requests CI/CD architecture](../img/qa_on_merge_requests_cicd_architecture.png) - -
-Show mermaid source -
+```mermaid
 graph LR
     A1 -.->|1. Triggers an omnibus-gitlab pipeline and wait for it to be done| A2
-    B2[`Trigger-qa` stage
`Trigger:qa-test` job] -.->|2. Triggers a gitlab-qa pipeline and wait for it to be done| A3
+    B2[`Trigger-qa` stage
`Trigger:qa-test` job] -.->|2. Triggers a gitlab-qa pipeline and wait for it to be done| A3
 
-subgraph gitlab-ce/ee pipeline
-    A1[`test` stage
`package-and-qa` job]
+subgraph "gitlab-ce/ee pipeline"
+    A1[`test` stage
`package-and-qa` job]
     end
 
-subgraph omnibus-gitlab pipeline
-    A2[`Trigger-docker` stage
`Trigger:gitlab-docker` job] -->|once done| B2
+subgraph "omnibus-gitlab pipeline"
+    A2[`Trigger-docker` stage
`Trigger:gitlab-docker` job] -->|once done| B2
     end
 
-subgraph gitlab-qa pipeline
-    A3>QA jobs run] -.->|3. Reports back the pipeline result to the `package-and-qa` job
and post the result  on the original commit tested| A1
+subgraph "gitlab-qa pipeline"
+    A3>QA jobs run] -.->|3. Reports back the pipeline result to the `package-and-qa` job
and post the result  on the original commit tested| A1
     end
-
-
+``` + 1. Developer triggers a manual action, that can be found in CE / EE merge requests. This starts a chain of pipelines in multiple projects. diff --git a/doc/development/testing_guide/img/qa_on_merge_requests_cicd_architecture.png b/doc/development/testing_guide/img/qa_on_merge_requests_cicd_architecture.png deleted file mode 100644 index 5b93a05db96..00000000000 Binary files a/doc/development/testing_guide/img/qa_on_merge_requests_cicd_architecture.png and /dev/null differ diff --git a/doc/development/testing_guide/img/review_apps_cicd_architecture.png b/doc/development/testing_guide/img/review_apps_cicd_architecture.png deleted file mode 100644 index 1ee28d3db91..00000000000 Binary files a/doc/development/testing_guide/img/review_apps_cicd_architecture.png and /dev/null differ diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md index d07a7045390..7843fc4c874 100644 --- a/doc/development/testing_guide/review_apps.md +++ b/doc/development/testing_guide/review_apps.md @@ -8,38 +8,33 @@ Review Apps are automatically deployed by each pipeline, both in ### CI/CD architecture diagram -![Review Apps CI/CD architecture](img/review_apps_cicd_architecture.png) - -
-Show mermaid source -
+```mermaid
 graph TD
     build-qa-image -.->|once the `prepare` stage is done| gitlab:assets:compile
     review-build-cng -->|triggers a CNG-mirror pipeline and wait for it to be done| CNG-mirror
     review-build-cng -.->|once the `test` stage is done| review-deploy
     review-deploy -.->|once the `review` stage is done| review-qa-smoke
 
-subgraph 1. gitlab-ce/ee `prepare` stage
+subgraph "1. gitlab-ce/ee `prepare` stage"
     build-qa-image
     end
 
-subgraph 2. gitlab-ce/ee `test` stage
+subgraph "2. gitlab-ce/ee `test` stage"
     gitlab:assets:compile -->|plays dependent job once done| review-build-cng
     end
 
-subgraph 3. gitlab-ce/ee `review` stage
-    review-deploy["review-deploy

Helm deploys the Review App using the Cloud
Native images built by the CNG-mirror pipeline.

Cloud Native images are deployed to the `review-apps-ce` or `review-apps-ee`
Kubernetes (GKE) cluster, in the GCP `gitlab-review-apps` project."]
+subgraph "3. gitlab-ce/ee `review` stage"
+    review-deploy["review-deploy

Helm deploys the Review App using the Cloud
Native images built by the CNG-mirror pipeline.

Cloud Native images are deployed to the `review-apps-ce` or `review-apps-ee`
Kubernetes (GKE) cluster, in the GCP `gitlab-review-apps` project."]
     end
 
-subgraph 4. gitlab-ce/ee `qa` stage
-    review-qa-smoke[review-qa-smoke

gitlab-qa runs the smoke suite against the Review App.]
+subgraph "4. gitlab-ce/ee `qa` stage"
+    review-qa-smoke[review-qa-smoke

gitlab-qa runs the smoke suite against the Review App.]
     end
 
-subgraph CNG-mirror pipeline
+subgraph "CNG-mirror pipeline"
     CNG-mirror>Cloud Native images are built];
     end
-
-
+``` ### Detailed explanation -- cgit v1.2.1 From 61e1a1492507ad2b77a48ef0a890de62caa590b3 Mon Sep 17 00:00:00 2001 From: Marcel Amirault Date: Mon, 5 Aug 2019 02:13:46 +0000 Subject: Expand markdown linting rules for docs MD002 - First header should be level 1 MD006 - Start bullets at beginning of line MD019 - No multiple spaces after header style MD022 - Headers surrounded by blank lines MD025 - Only 1 level 1 header MD028 - No blank lines within blockquote MD038 - Spaces inside code span elements --- .../testing_guide/end_to_end/quick_start_guide.md | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md index 14a169dcc1d..e1df8be8b6f 100644 --- a/doc/development/testing_guide/end_to_end/quick_start_guide.md +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -110,7 +110,7 @@ end ``` > Notice that the test itself is simple. The most challenging part is the creation of the application state, which will be covered later. - +> > The exemplified test case's MVC is not enough for the change to be merged, but it helps to build up the test logic. The reason is that we do not want to use locators directly in the tests, and tests **must** use [Page Objects] before they can be merged. This way we better separate the responsibilities, where the Page Objects encapsulate elements and methods that allow us to interact with pages, while the spec files describe the test cases in more business-related language. Below are the steps that the test covers: @@ -211,7 +211,7 @@ A pre-condition for the entire test suite is defined in the `before :context` bl > For our test suite, due to the need of the tests being completely independent of each other, we won't use the `before :context` block. The `before :context` block would make the tests dependent on each other because the first test changes the label of the issue, and the second one depends on the `'animal::fox'` label being set. -> **Tip:** In case of a test suite with only one `it` block it's ok to use only the `before` block (see below) with all the test's pre-conditions. +TIP: **Tip:** In case of a test suite with only one `it` block it's ok to use only the `before` block (see below) with all the test's pre-conditions. #### `before` @@ -274,11 +274,11 @@ end In the `before` block we create all the application state needed for the tests to run. We do that by using the `Runtime::Browser.visit` method to go to the login page, by performing a `sign_in_using_credentials` from the `Login` Page Object, by fabricating resources via APIs (`issue`, and `Resource::Label`), and by using the `issue.visit!` to visit the issue page. > A project is created in the background by creating the `issue` resource. - +> > When creating the [Resources], notice that when calling the `fabricate_via_api` method, we pass some attribute:values, like `title`, and `labels` for the `issue` resource; and `project` and `title` for the `label` resource. - +> > What's important to understand here is that by creating the application state mostly using the public APIs we save a lot of time in the test suite setup stage. - +> > Soon we will cover the use of the already existing resources' methods and the creation of your own `fabricate_via_api` methods for resources where this is still not available, but first, let's optimize our implementation. ### 6. Optimization @@ -362,7 +362,7 @@ First, in the [issue resource](https://gitlab.com/gitlab-org/gitlab-ee/blob/d358 Add the following `attribute :id` and `attribute :labels` right above the [`attribute :title`](https://gitlab.com/gitlab-org/gitlab-ee/blob/d3584e80b4236acdf393d815d604801573af72cc/qa/qa/resource/issue.rb#L15). > This line is needed to allow for the issue fabrication, and for labels to be automatically added to the issue when fabricating it via API. - +> > We add the attributes above the existing attribute to keep them alphabetically organized. Then, let's initialize an instance variable for labels to allow an empty array as default value when such information is not passed during the resource fabrication, since this optional. [Between the attributes and the `fabricate!` method](https://gitlab.com/gitlab-org/gitlab-ee/blob/1a1f1408728f19b2aa15887cd20bddab7e70c8bd/qa/qa/resource/issue.rb#L18), add the following: @@ -437,7 +437,7 @@ By defining the `resource_web_url(resource)` method, we override the one from th By defining the `api_get_path` method, we **would** allow for the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to get a single label, but since there's no path available for that in the publich API, we raise a `NotImplementedError` instead. -By defining the `api_post_path` method, we allow for the [`ApiFabricator `](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to create a new label in a specific project. +By defining the `api_post_path` method, we allow for the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to create a new label in a specific project. By defining the `api_post_body` method, we we allow for the [`ApiFabricator.api_post`](https://gitlab.com/gitlab-org/gitlab-ee/blob/a9177ca1812bac57e2b2fa4560e1d5dd8ffac38b/qa/qa/resource/api_fabricator.rb#L68) method to know which data to send when making the `POST` request. @@ -580,7 +580,7 @@ filter_output = search_field_tag search_id, nil, class: "dropdown-input-field", > `data-qa-*` data attributes and CSS classes starting with `qa-` are used solely for the purpose of QA and testing. > By defining these, we add **testability** to the application. - +> > When defining a data attribute like: `qa_selector: 'labels_block'`, it should match the element definition: `element :labels_block`. We use a [sanity test](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/qa/page#how-did-we-solve-fragile-tests-problem) to check that defined elements have their respective selectors in the specified views. #### Updates in the `QA::Page::Base` class @@ -599,8 +599,6 @@ This method receives an element (`name`) and the `keys` that it will send to tha As you might remember, in the Issue Page Object we call this method like this: `send_keys_to_element(:dropdown_input_field, [label, :enter])`. -___ - With that, you should be able to start writing end-to-end tests yourself. *Congratulations!* [Page Objects]: page_objects.md -- cgit v1.2.1 From 60dfca155c205a0fad1cac367f7225100cb555eb Mon Sep 17 00:00:00 2001 From: Marcel Amirault Date: Mon, 12 Aug 2019 04:23:01 +0000 Subject: Add more rules to markdown lint check Adds MD010 (Hard tabs), MD012 (blank lines), MD029 (ordered list prefix), MD030 (spaces after list markers), and fixes remaining docs that were failing these tests --- doc/development/testing_guide/end_to_end/index.md | 1 - doc/development/testing_guide/end_to_end/page_objects.md | 5 ++--- 2 files changed, 2 insertions(+), 4 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md index 882e2230636..d6b944a3e74 100644 --- a/doc/development/testing_guide/end_to_end/index.md +++ b/doc/development/testing_guide/end_to_end/index.md @@ -83,7 +83,6 @@ subgraph "gitlab-qa pipeline" end ``` - 1. Developer triggers a manual action, that can be found in CE / EE merge requests. This starts a chain of pipelines in multiple projects. diff --git a/doc/development/testing_guide/end_to_end/page_objects.md b/doc/development/testing_guide/end_to_end/page_objects.md index 52957d1a1ab..47e58a425fd 100644 --- a/doc/development/testing_guide/end_to_end/page_objects.md +++ b/doc/development/testing_guide/end_to_end/page_objects.md @@ -105,7 +105,7 @@ code but **this is deprecated** in favor of the above method for two reasons: view 'app/views/my/view.html.haml' do ### Good ### - + # Implicitly require the CSS selector `[data-qa-selector="logout_button"]` to be present in the view element :logout_button @@ -152,10 +152,9 @@ Things to note: - The name of the element and the qa_selector must match and be snake_cased - If the element appears on the page unconditionally, add `required: true` to the element. See [Dynamic element validation](dynamic_element_validation.md) -- You may see `.qa-selector` classes in existing Page Objects. We should prefer the [`data-qa-selector`](#data-qa-selector-vs-qa-selector) +- You may see `.qa-selector` classes in existing Page Objects. We should prefer the [`data-qa-selector`](#data-qa-selector-vs-qa-selector) method of definition over the `.qa-selector` CSS class - ### `data-qa-selector` vs `.qa-selector` > Introduced in GitLab 12.1 -- cgit v1.2.1 From dd84b9a03d24d27edafc861a988239de30f08a46 Mon Sep 17 00:00:00 2001 From: Giorgenes Gelatti Date: Tue, 13 Aug 2019 10:41:52 +1000 Subject: Mention focus: true in the specs --- doc/development/testing_guide/best_practices.md | 1 + 1 file changed, 1 insertion(+) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index 448d9fd01c4..9d6792e9139 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -70,6 +70,7 @@ bundle exec rspec spec/[path]/[to]/[spec].rb - On `before` and `after` hooks, prefer it scoped to `:context` over `:all` - When using `evaluate_script("$('.js-foo').testSomething()")` (or `execute_script`) which acts on a given element, use a Capyabara matcher beforehand (e.g. `find('.js-foo')`) to ensure the element actually exists. +- Use `focus: true` to isolate parts of the specs you want to run. [four-phase-test]: https://robots.thoughtbot.com/four-phase-test -- cgit v1.2.1 From 5a574883f95373e13f663568eb4710c9d69d00d6 Mon Sep 17 00:00:00 2001 From: Nick Thomas Date: Mon, 12 Aug 2019 11:29:10 +0100 Subject: Remove MySQL references from development docs I noticed the doc/development/testing_guide/best_practices.md still referenced the `[run mysql]` tags, etc. They no longer work, so I removed them, then realised I had better clean up the rest of doc/development ! --- doc/development/testing_guide/best_practices.md | 10 ---------- doc/development/testing_guide/ci.md | 1 - doc/development/testing_guide/flaky_tests.md | 4 ++-- 3 files changed, 2 insertions(+), 13 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index 448d9fd01c4..a505d3c79a2 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -15,16 +15,6 @@ manifest themselves within our code. When designing our tests, take time to revi our test design. We can find some helpful heuristics documented in the Handbook in the [Test Design](https://about.gitlab.com/handbook/engineering/quality/guidelines/test-engineering/test-design/) section. -## Run tests against MySQL - -By default, tests are only run against PostgreSQL, but you can run them on -demand against MySQL by following one of the following conventions: - -| Convention | Valid example | -|:----------------------|:-----------------------------| -| Include `mysql` in your branch name | `enhance-mysql-support` | -| Include `[run mysql]` in your commit message | `Fix MySQL support

[run mysql]` | - ## Test speed GitLab has a massive test suite that, without [parallelization], can take hours diff --git a/doc/development/testing_guide/ci.md b/doc/development/testing_guide/ci.md index 87d48726268..d9f66a827de 100644 --- a/doc/development/testing_guide/ci.md +++ b/doc/development/testing_guide/ci.md @@ -39,7 +39,6 @@ slowest test files and try to improve them. ## CI setup -- On CE and EE, the test suite runs both PostgreSQL and MySQL. - Rails logging to `log/test.log` is disabled by default in CI [for performance reasons][logging]. To override this setting, provide the `RAILS_ENABLE_TEST_LOG` environment variable. diff --git a/doc/development/testing_guide/flaky_tests.md b/doc/development/testing_guide/flaky_tests.md index 931cbc51cae..eb0bf6fc563 100644 --- a/doc/development/testing_guide/flaky_tests.md +++ b/doc/development/testing_guide/flaky_tests.md @@ -35,8 +35,8 @@ Once a test is in quarantine, there are 3 choices: Quarantined tests are run on the CI in dedicated jobs that are allowed to fail: -- `rspec-pg-quarantine` and `rspec-mysql-quarantine` (CE & EE) -- `rspec-pg-quarantine-ee` and `rspec-mysql-quarantine-ee` (EE only) +- `rspec-pg-quarantine` (CE & EE) +- `rspec-pg-quarantine-ee` (EE only) ## Automatic retries and flaky tests detection -- cgit v1.2.1 From 90170ab1a7fefd3d2ce81d81ed9df7bab7eb93c3 Mon Sep 17 00:00:00 2001 From: Thong Kuah Date: Sat, 17 Aug 2019 21:53:46 +0000 Subject: Remove reference to old lib/api/ci dir This directory no longer exists (since 2017) --- doc/development/testing_guide/testing_levels.md | 1 - 1 file changed, 1 deletion(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index e1ce4d3b7d1..6c3a3171d39 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -66,7 +66,6 @@ They're useful to test permissions, redirections, what view is rendered etc. | `app/controllers/` | `spec/controllers/` | RSpec | | | `app/mailers/` | `spec/mailers/` | RSpec | | | `lib/api/` | `spec/requests/api/` | RSpec | | -| `lib/ci/api/` | `spec/requests/ci/api/` | RSpec | | | `app/assets/javascripts/` | `spec/javascripts/`, `spec/frontend/` | Karma & Jest | More details in the [Frontend Testing guide](frontend_testing.md) section. | ### About controller tests -- cgit v1.2.1 From c2c86e35149b20b4dc3af083cc16ac42d50b3e58 Mon Sep 17 00:00:00 2001 From: Luke Duncalfe Date: Thu, 15 Aug 2019 16:56:04 +1200 Subject: Add `be_like_time` matcher to Testing Styleguide --- doc/development/testing_guide/best_practices.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index 448d9fd01c4..c7e58f58c61 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -454,6 +454,19 @@ complexity of RSpec expectations.They should be placed under a certain type of specs only (e.g. features, requests etc.) but shouldn't be if they apply to multiple type of specs. +#### `be_like_time` + +Time returned from a database can differ in precision from time objects +in Ruby, so we need flexible tolerances when comparing in specs. We can +use `be_like_time` to compare that times are within one second of each +other. + +Example: + +```ruby +expect(metrics.merged_at).to be_like_time(time) +``` + #### `have_gitlab_http_status` Prefer `have_gitlab_http_status` over `have_http_status` because the former -- cgit v1.2.1 From 3bc0a1185cdaa50c9cf8b2ffb30cd89442f8b2bc Mon Sep 17 00:00:00 2001 From: Marcel Amirault Date: Mon, 19 Aug 2019 23:41:23 +0000 Subject: Standardize markdown in dev and workflow Delete trailing whitespace, fix blockquotes, fix note boxes, with indentation, clean up tables, etc. --- doc/development/testing_guide/review_apps.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md index 7843fc4c874..11449712a04 100644 --- a/doc/development/testing_guide/review_apps.md +++ b/doc/development/testing_guide/review_apps.md @@ -255,8 +255,8 @@ that a machine will hit the "too many mount points" problem in the future. thousands of unused Docker images.** > We have to start somewhere and improve later. Also, we're using the - CNG-mirror project to store these Docker images so that we can just wipe out - the registry at some point, and use a new fresh, empty one. + > CNG-mirror project to store these Docker images so that we can just wipe out + > the registry at some point, and use a new fresh, empty one. **How do we secure this from abuse? Apps are open to the world so we need to find a way to limit it to only us.** -- cgit v1.2.1 From 49ea1a64c870fcc78ce01fd60cabfc5cc4f39349 Mon Sep 17 00:00:00 2001 From: Winnie Hellmann Date: Tue, 20 Aug 2019 01:43:13 +0000 Subject: Combine old and new frontend testing guide (docs) --- doc/development/testing_guide/frontend_testing.md | 359 ++++++++++++++++++++++ 1 file changed, 359 insertions(+) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index 2985278cc92..52ac746716c 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -593,6 +593,365 @@ end [capybara]: https://github.com/teamcapybara/capybara [jasmine]: https://jasmine.github.io/ +## Overview of Frontend Testing Levels + +Tests relevant for frontend development can be found at the following places: + +- `spec/javascripts/` which are run by Karma (command: `yarn karma`) and contain + - [frontend unit tests](#frontend-unit-tests) + - [frontend component tests](#frontend-component-tests) + - [frontend integration tests](#frontend-integration-tests) +- `spec/frontend/` which are run by Jest (command: `yarn jest`) and contain + - [frontend unit tests](#frontend-unit-tests) + - [frontend component tests](#frontend-component-tests) + - [frontend integration tests](#frontend-integration-tests) +- `spec/features/` which are run by RSpec and contain + - [feature tests](#feature-tests) + +All tests in `spec/javascripts/` will eventually be migrated to `spec/frontend/` (see also [#52483](https://gitlab.com/gitlab-org/gitlab-ce/issues/52483)). + +In addition, there used to be feature tests in `features/`, run by Spinach. +These were removed from the codebase in May 2018 ([#23036](https://gitlab.com/gitlab-org/gitlab-ce/issues/23036)). + +See also [Notes on testing Vue components](../fe_guide/vue.html#testing-vue-components). + +### Frontend unit tests + +Unit tests are on the lowest abstraction level and typically test functionality that is not directly perceivable by a user. + +#### When to use unit tests + +
+ exported functions and classes + Anything that is exported can be reused at various places in a way you have no control over. + Therefore it is necessary to document the expected behavior of the public interface with tests. +
+ +
+ Vuex actions + Any Vuex action needs to work in a consistent way independent of the component it is triggered from. +
+ +
+ Vuex mutations + For complex Vuex mutations it helps to identify the source of a problem by separating the tests from other parts of the Vuex store. +
+ +#### When *not* to use unit tests + +
+ non-exported functions or classes + Anything that is not exported from a module can be considered private or an implementation detail and doesn't need to be tested. +
+ +
+ constants + Testing the value of a constant would mean to copy it. + This results in extra effort without additional confidence that the value is correct. +
+ +
+ Vue components + Computed properties, methods, and lifecycle hooks can be considered an implementation detail of components and don't need to be tested. + They are implicitly covered by component tests. + The official Vue guidelines suggest the same. +
+ +#### What to mock in unit tests + +
+ state of the class under test + Modifying the state of the class under test directly rather than using methods of the class avoids side-effects in test setup. +
+ +
+ other exported classes + Every class needs to be tested in isolation to prevent test scenarios from growing exponentially. +
+ +
+ single DOM elements if passed as parameters + For tests that only operate on single DOM elements rather than a whole page, creating these elements is cheaper than loading a whole HTML fixture. +
+ +
+ all server requests + When running frontend unit tests, the backend may not be reachable. + Therefore all outgoing requests need to be mocked. +
+ +
+ asynchronous background operations + Background operations cannot be stopped or waited on, so they will continue running in the following tests and cause side effects. +
+ +#### What *not* to mock in unit tests + +
+ non-exported functions or classes + Everything that is not exported can be considered private to the module and will be implicitly tested via the exported classes / functions. +
+ +
+ methods of the class under test + By mocking methods of the class under test, the mocks will be tested and not the real methods. +
+ +
+ utility functions (pure functions, or those that only modify parameters) + If a function has no side effects because it has no state, it is safe to not mock it in tests. +
+ +
+ full HTML pages + Loading the HTML of a full page slows down tests, so it should be avoided in unit tests. +
+ +### Frontend component tests + +Component tests cover the state of a single component that is perceivable by a user depending on external signals such as user input, events fired from other components, or application state. + +#### When to use component tests + +- Vue components + +#### When *not* to use component tests + +
+ Vue applications + Vue applications may contain many components. + Testing them on a component level requires too much effort. + Therefore they are tested on frontend integration level. +
+ +
+ HAML templates + HAML templates contain only Markup and no frontend-side logic. + Therefore they are not complete components. +
+ +#### What to mock in component tests + +
+ DOM + Operating on the real DOM is significantly slower than on the virtual DOM. +
+ +
+ properties and state of the component under test + Similarly to testing classes, modifying the properties directly (rather than relying on methods of the component) avoids side-effects. +
+ +
+ Vuex store + To avoid side effects and keep component tests simple, Vuex stores are replaced with mocks. +
+ +
+ all server requests + Similar to unit tests, when running component tests, the backend may not be reachable. + Therefore all outgoing requests need to be mocked. +
+ +
+ asynchronous background operations + Similar to unit tests, background operations cannot be stopped or waited on, so they will continue running in the following tests and cause side effects. +
+ +
+ child components + Every component is tested individually, so child components are mocked. + See also shallowMount() +
+ +#### What *not* to mock in component tests + +
+ methods or computed properties of the component under test + By mocking part of the component under test, the mocks will be tested and not the real component. +
+ +
+ functions and classes independent from Vue + All plain JavaScript code is already covered by unit tests and needs not to be mocked in component tests. +
+ +### Frontend integration tests + +Integration tests cover the interaction between all components on a single page. +Their abstraction level is comparable to how a user would interact with the UI. + +#### When to use integration tests + +
+ page bundles (index.js files in app/assets/javascripts/pages/) + Testing the page bundles ensures the corresponding frontend components integrate well. +
+ +
+ Vue applications outside of page bundles + Testing Vue applications as a whole ensures the corresponding frontend components integrate well. +
+ +#### What to mock in integration tests + +
+ HAML views (use fixtures instead) + Rendering HAML views requires a Rails environment including a running database which we cannot rely on in frontend tests. +
+ +
+ all server requests + Similar to unit and component tests, when running component tests, the backend may not be reachable. + Therefore all outgoing requests need to be mocked. +
+ +
+ asynchronous background operations that are not perceivable on the page + Background operations that affect the page need to be tested on this level. + All other background operations cannot be stopped or waited on, so they will continue running in the following tests and cause side effects. +
+ +#### What *not* to mock in integration tests + +
+ DOM + Testing on the real DOM ensures our components work in the environment they are meant for. + Part of this will be delegated to cross-browser testing. +
+ +
+ properties or state of components + On this level, all tests can only perform actions a user would do. + For example to change the state of a component, a click event would be fired. +
+ +
+ Vuex stores + When testing the frontend code of a page as a whole, the interaction between Vue components and Vuex stores is covered as well. +
+ +### Feature tests + +In contrast to [frontend integration tests](#frontend-integration-tests), feature tests make requests against the real backend instead of using fixtures. +This also implies that database queries are executed which makes this category significantly slower. + +See also the [RSpec testing guidelines](../testing_guide/best_practices.md#rspec). + +#### When to use feature tests + +- Use cases that require a backend and cannot be tested using fixtures. +- Behavior that is not part of a page bundle but defined globally. + +#### Relevant notes + +A `:js` flag is added to the test to make sure the full environment is loaded. + +```js +scenario 'successfully', :js do + sign_in(create(:admin)) +end +``` + +The steps of each test are written using capybara methods ([documentation](https://www.rubydoc.info/gems/capybara)). + +Bear in mind XHR calls might require you to use `wait_for_requests` in between steps, like so: + +```rspec +find('.form-control').native.send_keys(:enter) + +wait_for_requests + +expect(page).not_to have_selector('.card') +``` + +## Test helpers + +### Vuex Helper: `testAction` + +We have a helper available to make testing actions easier, as per [official documentation](https://vuex.vuejs.org/guide/testing.html): + +```js +testAction( + actions.actionName, // action + { }, // params to be passed to action + state, // state + [ + { type: types.MUTATION}, + { type: types.MUTATION_1, payload: {}}, + ], // mutations committed + [ + { type: 'actionName', payload: {}}, + { type: 'actionName1', payload: {}}, + ] // actions dispatched + done, +); +``` + +Check an example in [spec/javascripts/ide/stores/actions_spec.jsspec/javascripts/ide/stores/actions_spec.js](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/javascripts/ide/stores/actions_spec.js). + +### Vue Helper: `mountComponent` + +To make mounting a Vue component easier and more readable, we have a few helpers available in `spec/helpers/vue_mount_component_helper`: + +- `createComponentWithStore` +- `mountComponentWithStore` + +Examples of usage: + +```js +beforeEach(() => { + vm = createComponentWithStore(Component, store); + + vm.$store.state.currentBranchId = 'master'; + + vm.$mount(); +}); +``` + +```js +beforeEach(() => { + vm = mountComponentWithStore(Component, { + el: '#dummy-element', + store, + props: { badge }, + }); +}); +``` + +Don't forget to clean up: + +```js +afterEach(() => { + vm.$destroy(); +}); +``` + +## Testing with older browsers + +Some regressions only affect a specific browser version. We can install and test in particular browsers with either Firefox or Browserstack using the following steps: + +### Browserstack + +[Browserstack](https://www.browserstack.com/) allows you to test more than 1200 mobile devices and browsers. +You can use it directly through the [live app](https://www.browserstack.com/live) or you can install the [chrome extension](https://chrome.google.com/webstore/detail/browserstack/nkihdmlheodkdfojglpcjjmioefjahjb) for easy access. +You can find the credentials on 1Password, under `frontendteam@gitlab.com`. + +### Firefox + +#### macOS + +You can download any older version of Firefox from the releases FTP server, : + +1. From the website, select a version, in this case `50.0.1`. +1. Go to the mac folder. +1. Select your preferred language, you will find the dmg package inside, download it. +1. Drag and drop the application to any other folder but the `Applications` folder. +1. Rename the application to something like `Firefox_Old`. +1. Move the application to the `Applications` folder. +1. Open up a terminal and run `/Applications/Firefox_Old.app/Contents/MacOS/firefox-bin -profilemanager` to create a new profile specific to that Firefox version. +1. Once the profile has been created, quit the app, and run it again like normal. You now have a working older Firefox version. + --- [Return to Testing documentation](index.md) -- cgit v1.2.1 From 3b7df66eef935046363c39d835b0a34d85b51584 Mon Sep 17 00:00:00 2001 From: Thong Kuah Date: Sun, 18 Aug 2019 08:57:24 +0000 Subject: Require request specs when testing N+1 for controllers --- doc/development/testing_guide/testing_levels.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/development/testing_guide') diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index e1ce4d3b7d1..6e1fdc765a1 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -63,7 +63,7 @@ They're useful to test permissions, redirections, what view is rendered etc. | Code path | Tests path | Testing engine | Notes | | --------- | ---------- | -------------- | ----- | -| `app/controllers/` | `spec/controllers/` | RSpec | | +| `app/controllers/` | `spec/controllers/` | RSpec | For N+1 tests, use [request specs](../query_recorder.md#use-request-specs-instead-of-controller-specs) | | `app/mailers/` | `spec/mailers/` | RSpec | | | `lib/api/` | `spec/requests/api/` | RSpec | | | `lib/ci/api/` | `spec/requests/ci/api/` | RSpec | | -- cgit v1.2.1