Opened 2 years ago

Closed 2 years ago

#33862 closed New feature (fixed)

Workflow to run the Django benchmarks when a pull request is labeled

Reported by: Deepak Dinesh Owned by: Deepak Dinesh
Component: Testing framework Version: dev
Severity: Normal Keywords: Workflow, GitHub actions
Cc: David Smith Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

One of the goals of the GSoC 22 Django benchmarking project was to provide a way for devs to run the benchmarks against their PR's to see the effect on the performance due to the changes made so that it can be improved before merging.

I have added a workflow that runs the benchmarks in smithdc1/django-asv against a pull request when it is labeled with the label benchmark. If the performance has not changed significantly, a pull request status message Benchmarking Result - BENCHMARKS NOT CHANGED SIGNIFICANTLY is added, if the performance has decreased a pull request status message Benchmarking Result - SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY. PERFORMANCE DECREASED is added.

The workflow only gets triggered when the PR's which are made after the workflow has been added are labeled.

Change History (6)

comment:1 by Carlton Gibson, 2 years ago

Cc: David Smith added
Component: UncategorizedTesting framework
Triage Stage: UnreviewedAccepted
Type: UncategorizedNew feature

Hey Deepak — Thanks.

Let's accept this. I'm not 100% clear immediately on the relation between the workflows across repos — but I'm guessing it's easier to see in code: if you open a PR on django/django for what needs adding, that would be great.

Maybe a scheduled trigger every X days (or every Monday, for example) would be useful too. 🤔

comment:2 by Deepak Dinesh, 2 years ago

A scheduled trigger has been added to smitdc1/django-asv, it currently runs once every day but I will change it to every Monday at a later stage.

comment:3 by Carlton Gibson, 2 years ago

Needs documentation: set
Patch needs improvement: set

Hey Deepak I'm going to mark this needing improvement*, whilst we resolve the details about the repo-interactions.
I'm looking at that now — will comment on the ASV repo.

*: And docs: we'll need to tell folks at least that the ASV project exists, and how to mark a PR for benchmarking (in the wiki at least, if not the contributing guide.)

comment:4 by Mariusz Felisiak, 2 years ago

Owner: changed from nobody to Deepak Dinesh

comment:5 by Carlton Gibson, 2 years ago

Needs documentation: unset
Patch needs improvement: unset
Triage Stage: AcceptedReady for checkin

comment:6 by Carlton Gibson <carlton@…>, 2 years ago

Resolution: fixed
Status: assignedclosed

In 982a9702:

Fixed #33862 -- Added workflow to run the ASV benchmarks for labeled PR.

Note: See TracTickets for help on using tickets.
Back to Top