feature/CSTACKEX-237: Pilot changes for scalebenmark framework - #85
feature/CSTACKEX-237: Pilot changes for scalebenmark framework#85piyush5netapp wants to merge 1 commit into
Conversation
sandeeplocharla
left a comment
sandeeplocharla
left a comment
There was a problem hiding this comment.
Overall, the pattern looks good but try to optimize the loggers and add more guardrails to the agent.
| python3 benchmark_storage_pool_concurrency.py --config config.yaml | ||
|
|
||
| # Concurrency matrix, iSCSI only, overriding the levels to run: | ||
| python3 benchmark_storage_pool_concurrency.py --config config.yaml --protocol iscsi --levels 2,5,10,20,30 |
There was a problem hiding this comment.
It should be enough to set config.yaml with all necessary combinations and use the fields appropriately in the code. Additional params might not be needed at this point.
| # Use the SAME --run-id across both scripts to combine 5.1.x + 6.1.x into one | ||
| # summary_<run_id>.csv / report (run sequential first, review it, then decide | ||
| # whether to proceed with concurrency under the same run id): | ||
| python3 benchmark_storage_pool_sequential.py --config config.yaml --run-id RUN-0001 |
There was a problem hiding this comment.
See if this run id can be randomized or create a pattern using timestamp or something?
| 6.1.1 Parallel pool creation | ||
| 6.1.2 Parallel pool deletion | ||
|
|
||
| See benchmark_storage_pool_sequential.py for the sequential scale matrix |
There was a problem hiding this comment.
Maybe add a rule in agents.md/custom-instruction file to only refer confluence but not add references to it in the code?
| else: | ||
| print(f" !! {result.error}") | ||
| if i in checkpoints: | ||
| total = sum(create_durations) |
There was a problem hiding this comment.
I guess the same statistics mean used in concurrent run script could be used here. Maybe include a rule to keep the code and libraries used consistent across the scripts.
| "total_time_sec": round(total, 3), "avg_time_sec": round(avg, 3), | ||
| "success_count": deleted_ok, "failure_count": idx - deleted_ok, "notes": "", | ||
| }) | ||
| print(f" >> checkpoint remaining={remaining}: total={total:.3f}s avg={avg:.3f}s/op " |
There was a problem hiding this comment.
Add a rule to keep the loggers also consistent across the scripts for better readability.
|
|
||
| def main(): | ||
| parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) | ||
| parser.add_argument("--config", default="config.yaml", help="Path to config YAML (default: config.yaml)") |
There was a problem hiding this comment.
As all the scripts seem to have same/similar args, it would be better to have a run.py file to which you would pass both the args and the name of the test you want to run. The respective script could be invoked from there while the additional args passed could be used to edit config file.
2 participants
Description
This PR...
Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
Bug Severity
Screenshots (if appropriate):
How Has This Been Tested?
How did you try to break this feature and the system with this change?