Babel bot ideas/discussion

[hzoo]

https://github.com/kubernetes/community/wiki/Roadmap:-Contributor-Experience

Bots

• eslintbot https://github.com/eslint/eslint/pulls
  • check CLA, add triage label to issues, validate commit messages, provide more contributor info
• mention-bot https://github.com/facebook/mention-bot
  • pings people for review
• kubernetes
  • k8s-merge-robot, k8s-ci-robot: (cla, triage, do-not-merge, size of PR, kind of pr, a lot of stuff!!)
• https://github.com/rust-lang/rust/
  • rust-highfive
• nodejs https://github.com/nodejs/github-bot

[hzoo]

bot

• logging failing tests (determining flaky tests)

Issues

• default label of "triage" like in eslint
• can close issues if awaiting response from user for more than x days (say 30,60 days) and auto open if they respond again, leave a message to explain that

PRs

• can label if new contributor (first pr ever, to project, etc) (or other stats or maybe not)
• can assign people via mention-bot (if it has an api)
• can re-ping collaborators if users are expecting a response and we haven't responded
• label the pkgs changed in a pr (pkg: babel-generator, etc)
• parse the pr table data to add some labels (semver, changelog data)
• read the failed tests and print as a message, possibly add extra info based on the error message (if linting rule, explain how to autofix/run tests, if a test explain how to fix the failing test, etc)
• maybe check 👍s of collabs in the bot comment and if > certain amount add approved label for an issue?
• can check if package.json is updated and add a label, etc
• run license-checker
• add a command to rerun tests! (we have a "retest" command at work)

[TheLarkInn]

What are bots usually written in. Could this be as simple as an express server listening on hooks from a repo?

[hzoo]

@DrewML set up something on AWS lambda yeah with hooks, although some could possibly run as a cron or as part of travis depending on the task

[DrewML]

Could this be as simple as an express server listening on hooks from a repo

Yes and no. For things that aren't scheduled tasks, it can. Having said that, there are some auth pains with that. You can't tell Github hooks to send you any secret in an API request, so anyone who discovers the URI to your host could send payloads masquerading as official events from Github. In theory, you could see if Github has an IP range that you could whitelist, but, meh ¯\(ツ)/¯

Github has an integration with the AWS push notification service, that you can use to trigger Lamdba functions for specific events. I ended up going this route for the Babel bot because it'll be the cheapest option (only paying for time used, and has a free tier), and we're not currently doing anything like OpenCollect, so this is all out of pocket.

For things like scheduled events (like sending warnings on issues after X days), likely just going to set up another Lambda function that is scheduled.

[TheLarkInn]

Awesome. I'd be willing to conform our repo to similar labels for PR statuses and etc. if it allows us to allow for more flexibility and usability across orgs. Or maybe a .yaml file that you set a label name for each event hook.

coverage-fail: 
  - "PR: Needs Tests"

Something that each project could drop into their repo and use the service across the board.

[DrewML]

@TheLarkInn Yeah! I think, for the CI failing tests stuff (adding the failures in a comment on the PR), I was going to break that out from babel-bot, since that's something pretty generic, and doesn't need a separate server to run (can just execute on the CI server).

[mxstbr]

DangerJS does many of the things you want to do here already! https://github.com/danger/danger-js

We use it over at styled-components to make our lives easier, see here for how we implemented it. This is what it does for us:

• Enforce CHANGELOG.md entries for each PR that touches .js files
• A change was made in StyledComponent.js that wasn't made in StyledNativeComponent.js – warn that this might have to be done
• Warn that changes to ThemeProvider.js, StyledComponent.js or StyledNativeComponent.js might be semver major changes if they touch the context stuff
• Don't let testing shortcuts be merged into master (fit, fdescribe, it.only, describe.only)
• Warn that a test must be added for the changes?
• Warn for big PR (> 500 lines)
• Congrats for version bump up 🎉
• Warn if an external contributor (a person who is not part of the organization) changes package.json
• Fail build if PR has no description

It's amazing, we love it! Note how many of these rules are very specific to the project, but how many are also very general and can be used by other projects – it's great. @orta has been really hard at work on it, and it's super useful. I've been a fan for a while now, and I'd love for the community to rally around that project.

Note that DangerJS is a groud-up rewrite/port of the more mature original Danger, which is written in Ruby and only let's you write your Dangerfile in Ruby

[xtuc]

Just an idea, but what about integrating stuff in https://github.com/facebook/mention-bot ?

The "Programmatic API" seems great.

[danharper]

You can't tell Github hooks to send you any secret in an API request, so anyone who discovers the URI to your host could send payloads masquerading as official events from Github.

@DrewML GitHub allow you to specify a secret key, which they use to HMAC the webhook body, and place the hash in the x-hub-signature header. To verify requests, you do the same process - HMAC the incoming body with the shared secret, and compare to the header.

https://developer.github.com/webhooks/securing/

Example in Node - the hexademical example

[DrewML]

@danharper That's awesome! Hadn't seen that before. Going to update the babel bot tonight, since that'll make the initial setup much less complicated (AWS SNS is a pita).

[DrewML]

Danger looks cool, but will really only work for a small subset of the things we'd like to accomplish, since it can only run in CI at the time of a PR.

[boneskull]

interested in this, might be able to help. cc @mikesherov

[mikesherov]

@boneskull, @hzoo is my coworker. If we can all get on same page, that'd be great!

[DrewML]

Start of the bot can be found here. Might be a good time to start breaking out this thread into individual issues on that repo for further discussion/implementation details.