Scenario tests design

https://blueprints.launchpad.net/manila/+spec/scenario-tests

Manila has good functional test coverage of its features, but narrow coverage of scenarios. And with addition of new features this coverage becomes even less.

问题描述

Drivers often lack features or implement them incorrectly and it’s very difficult for humans to notice this in code review. Tests ensure that drivers which lack features must explicitly skip tests to get a passing result, and the non-skipped tests ensure that the drivers implement features correctly. Documentation for how features should work is often sparse and driver authors don’t always know what is the correct behavior when implementing a driver feature. Tests allow driver authors to code the feature to pass the test, which simplifies the driver author’s job.

用例

Consider the case when new manila deployment becomes ready. And one wants to verify that main user use cases work in general. Having automatic scenario tests one could test his deployment much faster than manually, as it is now. Also, it could be used in CI systems to test share drivers continuously.

提议的变更

It is proposed to get agreement on scenario tests design (this spec) and implement them in manila plugin for tempest. After that these tests could be used in CI systems as well as on customer deployments. This spec covers only existing features in manila as of last available (Newton) release. All newly added features should be covered separately.

Prerequisites for scenarios

• Depending on share driver mode, it can be required to create share-network with or without security-services.

• Depending on protocol, its versions and access type, “mount” operations should be defined explicitly. Scenario tests assume it as predefined and known. Hence, scenarios do not include difference between access type (IP, User, Cert). Due to this, scenario tests should be data driven by “access_type”, “access_proto”, “access_level” and “mount command with all expected options”.

• 粗体文字 depend on specific implementations and can be unsupported by share backends.

• 方括号内的文字 depend on share driver configuration and may be optional.

• All user machines are separate VMs from share-nodes. They should have network connectivity with share host. User VMs should be built with image that has shared file systems clients.

• Share host should have open ports for SSH protocol and protocols of shared file systems.

Here is list of scenarios for implementation

• 1) Share access and file operations (Partially implemented)

• 2) Share access with multiple guests (Partially implemented)

• 3) Relationships between source shares and child shares (Implemented)

• 4) Create/extend share and write data (Implemented)

• 5) Create/shrink share and write data (Implemented)

• 6) Create/manage share and write data (Implemented)

• 7) Create/manage share and snapshot and write data (Partially implemented)

• 8) Replicate ‘writable’ share and write data (Not yet implemented)

• 9) Replicate and promote ‘readable’ share and write data (Not yet implemented)

• 10) Replicate and promote ‘dr’ share and write data (Not yet implemented)

• 11) Get a snapshot of a replicated share (Not yet implemented)

• 12) Migrate share and write data (Implemented)

1) Share access and file operations

Driver mode: any

Involved APIs

• [创建 share network]

• [创建 share 类型]

• 创建 share

• 允许 share 访问

• 拒绝 share 访问

• 删除 share

注意

Implementation Status

This test case extends manila_tempest_tests.tests.scenario .test_share_basic_ops.ShareBasicOpsBase#test_write_with_ro_access

Scenario 1 steps

Step	操作	结果
1	创建 user VM (UVM)	ok, created
2	创建 share (S)	ok, created
3	SSH to UVM	ok, connected
4	尝试 mount S to UVM	fail, access denied
5	提供 RO access to S	ok, provided
6	尝试 mount S to UVM	ok, mounted
7	尝试创建文件 on S	fail, access denied
8	Unmount S from UVM	ok, unmounted
9	移除 RO access from S	ok, removed
10	尝试 mount S to UVM	fail, access denied
11	提供 RW access to S	ok, provided
12	尝试 mount S to UVM	ok, mounted
13	尝试写入文件 to S	ok, written
14	尝试读取文件 from S	ok, read
15	尝试删除文件 on S	ok, deleted
16	Unmount S from UVM	ok, unmounted
17	删除 S	ok, deleted
18	尝试 mount S	fail, not found
19	删除 UVM	ok, deleted

2) Share access with multiple guests

Driver mode: any

Involved APIs

• [创建 share network]

• [创建 share 类型]

• 创建 share

• 允许 share 访问

• 删除 share

注意

Implementation Status

This test case extends manila_tempest_tests.tests.scenario .test_share_basic_ops.ShareBasicOpsBase#test_read_write_two_vms

Scenario 2 steps

Step	操作	结果
1	创建 UVM1	ok, created
2	创建 UVM2	ok, created
3	创建 share S	ok, created
4	添加 RW access to UVM1	ok, added
5	SSH to UVM1	ok, connected
6	尝试 mount S from UVM1	ok, mounted
7	SSH to UVM2	ok, connected
8	尝试 mount S from UVM2	fail, access denied
9	添加 RW access for UVM2	ok, added
10	尝试 mount S from UVM2	ok, two VMs have it mounted at once.
11	创建测试文件 in mounted share from UVM1	ok, created. Available from UVM2
12	写入数据 to test file from UVM2	ok, written. Available from UVM1 too
13	Unmount S on UVM1	ok, unmounted
14	Unmount S on UVM2	ok, unmounted
15	删除 UVM1	ok, deleted
16	删除 UVM2	ok, deleted
17	删除 S	ok, deleted

3) Relationships between source shares and child shares

Driver mode: any

Involved APIs

• [创建 share network]

• [创建 share 类型]

• 创建 share

• 允许 share 访问

• 创建 share snapshot

• 删除 share snapshot

• 删除 share

注意

Implementation Status

This test case has been implemented as manila_tempest_tests.tests .scenario.test_share_basic_ops .ShareBasicOpsBase#test_write_data_to_share_created_from_snapshot

Scenario 3 steps

Step	操作	结果
1	创建 UVM	ok, created
2	创建 share S1	ok, created
3	提供 RW access to S1	ok, provided
4	SSH to UVM	ok, connected
5	尝试 mount S1 to UVM	ok, mounted
6	创建 “file1”	ok, created
7	创建 snapshot SS1 from S1	ok, created
8	创建 “file2” in share S1	ok, created. We expect that snapshot will not contain any data created after snapshot creation.
9	创建 share S2 from SS1	ok, created
10	尝试 mount S2	fail, access denied. We test that child share did not get access rules from parent share.
11	提供 RW access to S2	ok, provided
12	尝试 mount S2	ok, mounted
13	列出文件 on S2	only “file1” exists
14	创建 file3 on S2	ok, file created
15	列出文件 on S1	two files exist - “file1” and “file2”
16	列出文件 on S2	two files exist - “file1” and “file3”
17	Unmount S1 and S2	ok, unmounted
18	删除 S2, then SS1, then S1, then UVM	ok, all deleted

4) Create/extend share and write data

Driver mode: any

Involved APIs

• [创建 share network]

• [创建 share 类型]

• 创建 share

• 允许 share 访问

• extend share

• 删除 share

注意

Implementation Status

This test case has been implemented as manila_tempest_tests.tests .scenario.test_share_extend.ShareExtendBase#test_create_extend_and_write

Scenario 4 steps

Step	操作	结果
1	创建 UVM	ok, created
2	创建 share S1 of 1Gb size	ok, created
3	提供 RW access to S1	ok, provided
4	SSH to UVM	ok, connected
5	尝试 mount S1 to UVM	ok, mounted
6	创建 “file1”	ok, created
7	填充 file1 with data as possible	size of a file does not exceed share size quota
8	扩展 share S1 to 2Gb	ok, extended
9	写入 additional data to file1	data written, size of a file does not exceed new share size quota and it is more than old one
10	Unmount S1	ok, unmounted
11	删除 share S1	ok, deleted
12	删除 UVM	ok, deleted

5) Create/shrink share and write data

Driver mode: any

Involved APIs

• [创建 share network]

• [创建 share 类型]

• 创建 share

• 允许 share 访问

• shrink share

• 删除 share

注意

Implementation Status

This test case has been implemented as manila_tempest_tests.tests .scenario.test_share_shrink.ShareShrinkBase#test_create_shrink_and_write

Scenario 5 steps

Step	操作	结果
1	创建 UVM	ok, created
2	创建 share S1 of 2Gb size	ok, created
3	提供 RW access to S1	ok, provided
4	SSH to UVM	ok, connected
5	尝试 mount S1 to UVM	ok, mounted
6	写入 some data for 2 Gb	ok, created
7	填充 file1 with data as possible	size of a file does not exceed share size quota
8	尝试 shrink share S1 to 1Gb	fail, possible data loss exception
9	删除 data for amount of 1 Gb	data deleted
10	Shrink share S1 to 1Gb	ok, shrinked
11	尝试 write data more than new size of 1 Gb	fail, cannot write
12	Unmount S1	ok, unmounted
13	删除 share S1	ok, deleted
14	删除 UVM	ok, deleted

6) Create/manage share and write data

Driver mode: any

Involved APIs

• [创建 share network]

• [创建 share 类型]

• 创建 share

• 允许 share 访问

• manage share

• unmanage share

• manage share again

• 删除 share

注意

Implementation Status

This test case has been partially implemented as manila_tempest_tests .tests.scenario.test_share_manage_unmanage .ShareManageUnmanageBase#test_create_manage_and_write . It currently tests only DHSS=False back end share drivers. To complete the implementation, this test case needs to support DHSS=True mode of share drivers. Support for managing shares with DHSS=True was added to Manila via API version 2.49. So this test must create a share network if [share]/multitenancy_enabled=True and the API version being tested is >= 2.49, and manage the share into the specific share network.

Scenario 6 steps

Step	操作	结果
1	创建 UVM	ok, created
2	创建 share S1 of 1Gb size	ok, created
3	提供 RW access to S1	ok, provided
4	SSH to UVM	ok, connected
5	尝试 mount S1 to UVM	ok, mounted
6	写入 some data	ok, written
7	Unmount S1	ok, unmounted
8	Unmanage share	ok, unmanaged
9	尝试 get share S1	fail, 404 code in response
10	Manage share S1	ok, managed.
11	提供 RW access to S1 again	ok, provided. We make sure that even if rule has existed on backend, we do not fail if explicitly try add it again after ‘manage’ operation.
12	尝试 mount S1 to UVM	ok, mounted. Previously created data still there.
13	Unmount S1	ok, unmounted
14	删除 share	ok, deleted
15	尝试 manage share again	fail, resource not found
16	删除 UVM	ok, deleted

7) Create/manage share and snapshot and write data

Driver mode: any

Involved APIs

• [创建 share network]

• [创建 share 类型]

• 创建 share

• 允许 share 访问

• 创建快照

• manage share

• unmanage share

• manage snapshot

• unmanage snapshot

• delete snapshot

• 删除 share

注意

Implementation Status

This test case is yet to be implemented.

Scenario 7 steps

Step	操作	结果
1	创建 UVM	ok, created
2	创建 share S1 of 1Gb size	ok, created
3	提供 RW access to S1	ok, provided
4	SSH to UVM	ok, connected
5	尝试 mount S1 to UVM	ok, mounted
6	写入 some data	ok, written
7	创建 snapshot SS1	ok, created
8	Unmanage snapshot SS1	ok, unmanaged
9	Unmanage share S1	ok, unmanaged
10	尝试 get share S1	fail, 404 code in response
11	Manage share S1	ok, managed.
12	提供 RW access to S1 again	ok, provided. We make sure that even if rule has existed on backend, we do not fail if explicitly try add it again after ‘manage’ operation.
13	尝试 mount S1 to UVM	ok, mounted. Previously created data still there.
14	Manage snapshot SS1	ok, managed
15	删除 snapshot SS1	ok, deleted
16	Unmount S1	ok, unmounted
17	删除 share S1	ok, deleted
18	尝试 manage share S1 again as S2, this should fail asynchronously since the resource is gone	S2 has a status set to ‘error’
19	删除 S2	ok, deleted
20	删除 UVM	ok, deleted

8) Replicate ‘writable’ share and write data

Driver mode: any

Involved APIs

• [创建 share network]

• [创建 share network subnets in different availability zones]

• [创建 share 类型]

• 创建 share

• 允许 share 访问

• 创建 replica

• 删除 replica

• 删除 share

注意

Implementation Status

This test case is yet to be implemented.

Scenario 8 steps

Step	操作	结果
1	创建 UVM1	ok, created
2	创建 share S1-R1 of 1Gb size	ok, created
3	提供 RW access to S1-R1	ok, provided
4	SSH to UVM1	ok, connected
5	尝试 mount S1-R1 to UVM1	ok, mounted
6	创建 file1	ok, created
7	创建 share replica S1-R2	ok, created
8	创建 UVM2	ok, created
9	SSH to UVM2	ok, connected
10	尝试 mount S1-R2 to UVM2	fail, access denied
11	尝试 mount S1-R2 to UVM1	ok,mounted. Same files exist.
12	提供 RW access to S1-R2	ok, provided
13	尝试 mount S1-R2 to UVM2	ok, mounted
14	创建 file2 in S1-R2	ok, created. S1-R1 has both files too.
15	创建 file3 in S1-R1	ok, created. Both replicas have three created files.
16	Unmount both replicas	ok, unmounted
17	删除 original replica S1-R1	ok, deleted. second and the only replica now still exists and has all files that were created.
18	删除 share S1	ok, deleted

9) Replicate and promote ‘readable’ share and write data

Driver mode: any

Involved APIs

• [创建 share network]

• [创建 share network subnets in different availability zones]

• [创建 share 类型]

• 创建 share

• 允许 share 访问

• 创建 replica

• promote replica

• 删除 replica

• 删除 share

注意

Implementation Status

This test case is yet to be implemented.

Scenario 9 steps

Step	操作	结果
1	创建 UVM1	ok, created
2	创建 share S1-R1 of 1Gb size	ok, created
3	提供 RW access to S1-R1	ok, provided
4	SSH to UVM1	ok, connected
5	尝试 mount S1-R1 to UVM1	ok, mounted
6	创建 file1	ok, created
7	创建 share replica S1-R2	ok, created
8	创建 UVM2	ok, created
9	SSH to UVM2	ok, connected
10	尝试 mount S1-R2 to UVM2	fail, access denied
11	尝试 mount S1-R2 to UVM1	ok, mounted. Same files exist.
12	提供 RW access to S1-R2	ok, provided
13	尝试 mount S1-R2 to UVM2	ok, mounted
14	尝试 create some file in S1-R2	fail, filesystem is RO only.
15	创建 file2 in S1-R1	ok, created. Both replicas have two created files.
16	Promote S1-R2 to active	ok, promoted. S1-R1 became RO.
17	创建 file3 in S1-R2	ok, created. S1-R1 has all files too.
18	尝试 create some file in S1-R1	fail, filesystem is RO
19	Unmount both replicas	ok, unmounted
20	删除 original (now RO) replica S1-R1	ok, deleted. Second and the only replica (active) now still exists and has all files that were created.
21	删除 share S1	ok, deleted

10) Replicate and promote ‘dr’ share and write data

Driver mode: any

Involved APIs

• [创建 share network]

• [创建 share network subnets in different availability zones]

• [创建 share 类型]

• 创建 share

• 允许 share 访问

• 创建 replica

• promote replica

• 删除 replica

• 删除 share

注意

Implementation Status

This test case is yet to be implemented.

场景 10 步骤

Step	操作	结果
1	创建 UVM	ok, created
2	创建共享 S1-R1	ok, created
3	提供 RW access to S1-R1	ok, provided
4	SSH to UVM	ok, connected
5	尝试将 S1-R1 挂载到 UVM	ok, mounted
6	创建 file1	ok, created
7	创建 share replica S1-R2	ok, created
8	卸载 S1-R1	ok, unmounted
9	提升 S1-R2	好的，已提升。S1-R1 变为仅‘dr’
10	尝试将 S1-R2 挂载到 UVM	好的，已挂载。‘file1’ 存在
11	创建 file2	ok, created
12	卸载 S1-R2	ok, unmounted
13	提升 S1-R1	好的，已提升。S1-R2 变为仅‘dr’。
14	尝试将 S1-R1 挂载到 UVM	好的，已挂载。文件 ‘file1’ 和 ‘file2’ 存在。
15	卸载 S1-R1	ok, unmounted
16	删除 S1-R2（当前仲备)	好的，已删除。
17	删除 share	ok, deleted

11) 获取复制共享的快照

Driver mode: any

Involved APIs

• [创建 share network]

• [创建 share network subnets in different availability zones]

• [创建 share 类型]

• 创建 share

• 允许 share 访问

• 创建快照

• 从快照创建共享

• 创建 replica

• promote replica

• 删除 replica

• delete snapshot

• 删除 share

注意

Implementation Status

This test case is yet to be implemented.

场景 11 步骤

Step	操作	结果
1	创建 UVM	ok, created
2	创建共享 S1-R1	ok, created
3	提供 RW access to S1-R1	ok, provided
4	SSH to UVM	ok, connected
5	尝试将 S1-R1 挂载到 UVM	ok, mounted
6	创建 ‘file1’	ok, created
7	创建 snapshot SS1	ok, created
8	创建副本 S1-R2	ok, created
9	创建 ‘file2’	ok, created
10	创建快照 SS2	ok, created
11	卸载 S1-R1	ok, unmounted
12	提升 S1-R2（对于非‘可写’复制类型）	好的，已提升
13	尝试将 S1-R2 挂载到 UVM	ok, mounted
15	删除 S1-R1	ok, deleted
16	从 SS2 创建共享 S2	ok, created
17	提供 RW access to S2	ok, provided
18	SSH to UVM	ok, connected
19	尝试将 S2 挂载到 UVM	好的，已挂载。所有创建的文件都存在
20	卸载 S2	ok, unmounted
21	删除 S2、SS2、SS1、S1	ok, deleted

12) 迁移共享并写入数据

Driver mode: any

Involved APIs

• [创建 share network]

• [创建 share 类型]

• 创建 share

• 允许 share 访问

• migration-start share

• migration-complete share

• 删除 share

注意

Implementation Status

此测试用例已实现为 manila_tempest_tests.tests .scenario.test_share_basic_ops.ShareBasicOpsBase#test_migration_files

场景 12 步骤

Step	操作	结果
1	创建 UVM	ok, created
2	创建 share S1 of 1Gb size	ok, created
3	提供 RW access to S1	ok, provided
4	SSH to UVM	ok, connected
5	尝试 mount S1 to UVM	ok, mounted
6	创建 file1	ok, created
7	卸载共享 S1	ok, unmounted
8	执行“migration-start”	好的，完成。1 个阶段已完成。
9	执行“migration-complete”	好的，共享实例只有一个 - 新的。它具有先前创建的 file1。
10	尝试 mount S1 to UVM	好的，已挂载。创建的 file1 存在
11	卸载共享 S1	ok, unmounted
12	删除 share S1	ok, deleted
13	删除 UVM	ok, deleted

备选方案

另一种选择是我们现在拥有的。要求手动测试每个共享驱动程序，并依赖于每个功能共享驱动程序实现详细文档的存在。

数据模型影响

无

REST API 影响

无

驱动程序影响

无

安全影响

无

通知影响

无

其他最终用户影响

最终用户将能够针对其 manila 部署运行场景测试，以测试各种功能的可行性。

性能影响

无

其他部署者影响

无

开发人员影响

无

实现

负责人

原始分配人

• vponomaryov

其他贡献者

• 我们邀请更多贡献者继续改进场景测试。将新的场景测试用例添加到 manila-tempest-plugin 不需要将测试用例描述添加到此规范中，但如果您希望对新的测试用例获得反馈，则鼓励这样做。

工作项

• 在 tempest 的 manila 插件中实现设计的场景测试。

依赖项

无

测试

预计所有第一方驱动程序以及第三方驱动程序都将在 CI 系统中通过此处设计的场景测试覆盖。由于场景测试涵盖了大量的可选功能，因此 CI 系统中应仅运行针对特定后端的适当场景测试。仅包含必需功能的场景（1-4）是每个共享驱动程序在 CI 系统中运行的必须条件。

文档影响

描述 tempest 中 manila 插件用法的文档 [1] 应扩展为包含场景测试的配置和用法细节。

参考资料

• [1] https://docs.openstack.org/developer/manila/devref/tempest_tests.html