Timeline and Checklist for Hack Weeks

This is a brief checklist for organizers of hack weeks, based on the supplementary materials of the hack week paper.

These are suggestions that we’ve found helpful in the past. Feel free to adapt and change according to your needs!

About 1-1.5 years in advance

Form a Scientific and Local Organizing Committee(s)

Define the different roles required for successful organization: e.g. building a website, coordinating tutorial leads, corresponding with participants, advertising the event, coordinating with local staff (about rooms, catering etc), facilitation during the event
Build a team that is distributed in expertise and interests across these roles
Elect a "cat herder in chief": a person who makes sure the team is on track, online calls get scheduled and reminders are sent out.
Make sure things get done:

Define all roles
Build a roadmap for completion of tasks
Assign tasks to team members

Establish rules for working together (e.g. a code of conduct among the committee)
Settle on team communication (asynchronous communication via slack/e-mail? Regular online calls?)

Objectives

Figure out your motivation for organizing a hack week. What drives the need for a hack week?

What are the objectives of a hack week? Set the objectives and goals:
What do you want participants to produce?
What do you want participants to have learned?
How/where/in what form do you want participants to connect with each other?
What other possible goals could you have during a hack week

2. Prepare for Feedback + Evaluation

Are there certain goals (or possibly requirements from a funding agency) with respect to evaluation of the workshop? Clarify those goals early on!
If any scientific research is planned in any way in relation to the hack week, this is the time to initiate the procedures for ethical research (depending on country/institution; in the US this is run by an Institutional Review Board)! In most countries, any human subject research must be cleared by the ethics board of the institution! Do this!

Location + Venue

Determine country/city for your hack week

Are there visa requirements for the country?
How easy/difficult is it to travel to/from this location?
Is the country the location is in restrictive in any way or unsafe for certain participants (e.g. participants of color, LGBTQ participants, participants from certain countries)
What is accommodation like in this country/city?
How good is the public transport in the chosen location?
How expensive are travel + accommodation + food in the proposed location? Is this financially accessible to students and participants from institutions with less funding?
What is the environment like with respect to your field? Is there a significant local presence (might not be important, but could be!)
Is there something for participants to do outside of the hack week (may be a plus or minus, depending on whether you want participants to work together all the time)
What is available in terms of families, child support, etc.?
Is the location known to have tech industry? This may only matter if you’re interested in connecting with local companies and/or ask them for funding.

Venue: Considerations for facilitating hacking

What are the venue’s costs? Is there local funding available that might come with the venue?
Are there any location-specific events going on around the venue during that time that might interfere with travel or the hackweek itself?
How large is the room? Probably want a room that’s at least one-third larger than technically needed for the number of people (so a hack week with 50 people should be in a room advertised for 65 or so)
Are there separate rooms, or one big one? Separate rooms might facilitate break-outs, but make it harder to keep the group together and allow for work across projects
What is the set-up for the room? Lecture theatre-style or tables? Theatre-stype is terrible for collaboration, so try not to spend (too much) time in a lecture theatre
Are components of the room moveable and can chairs/tables be reconfigured? Reconfiguring helps with flexible group formation and interactive work.
How many white boards/chalk boards are there? Depending on group size, probably want more than 3
How many projectors are there? You need at least 1 that’s visible from the entire room.
Power outlets! Need lots of them
Internet: high-speed internet is crucial for many data-related tasks

Venue: General Considerations

Does the venue require an application (e.g. the Lorentz Centre)? If so, check application procedures + deadlines >1 year in advance!
Is the space accessible? Can people with physical disabilities move freely (e.g. participants with wheelchairs, blind participants, …)? Are there elevators or only stairs?
Are bathrooms gender-neutral or can they be made gender-neutral? Is there administrative support in terms of practical organization (e.g. handling reimbursement claims, ordering lunch/coffee/rooms, assisting with accommodation, ...)? This can be a huge help to the SOC!
How far away is the venue from available accommodation?
Is there coffee/tea?
Advertise location/venue as soon as possible once you know the dates and location (e.g. via the AAS Astrostatistics Working Group, this list of astronomy meetings, the Astrostatistics + Astroinformatics Portal, e-mail lists, local universities, everyone you know)

Funding

A hack week requires funding for a number of different costs: rooms (not always), lunch, coffee, travel funding for some of the participants, potentially salaries for organizers and especially administrative staff, and of course some incidentals.
Do rooms need to be paid for extra? This is often the case e.g. in the US! Check this!
What are available governmental funding sources? (multi-)national agencies (NIH, NASA, ERC, …): this funding probably needs an application >1 year in advance of the event
Are there private foundations that might be willing to provide funding (e.g. Moore and Sloan Foundations)? Make contact + check procedures >1 year in advance
Are there other local funding sources available (e.g. does the university/venue have funding or come with funding attached to the workshop)?
Are there private companies you can ask for funding? Check the local tech scene! In the past, local offices of Google, IBM, Galvanize, and GitHub have all provided funding to AstroHackWeek
How much funding do you intend to reserve for travel grants to participants?
How does your funding from different sources match up with different funding needs (sometimes, funding sources come with conditions for what can/cannot be funded)?
How much are you likely to charge for registration? Charging a small amount for registration helps participants buy into coming (people are more likely to attend if they’ve put down money for it), but charging a large amount will make the meeting inaccessible
Check how to collect registration fees! In most countries, this may require an entity that can deal with the tax implications of having people give you money! If at all possible, try to convince the hosting venue or your university to deal with this!
Is there a designated fiscal person who can handle travel grant distributions and handling travel arrangement for any out of town instructors? Maybe this person could be added to the organizing committee as well.

About 1 year to 6 months in advance

Venue

Identify and book (at minimum, place a hold on) a venue that best meets your hackweek’s needs. This is crucial to avoid scrambling for a space close to the event. It is also often a challenge finding a venue that is available for back-to-back days of the hackweek.

Website

Prepare a Website: this should be done early on, definitely as soon as the hack week is announced.

What? Why? Where? How?
Who are the organizers?
Who is speaking (if you know it already)?
What is the schedule (if you know it already)?
Local information: travel + transport, hotel(s), food options
Contact information for the SOC
Information about live stream, code of conduct
(Link to application form, once it’s live)

Set up an e-mail address as point of contact for potential participants
Set up a mailing list for announcements that interested researchers can sign up for.

Announce the event

Where are typical venues for announcing events in your community?
Prepare e-mail to colleagues and mailing lists. In announcement emails, provide info on what a hackweek is, what your hackweek aims are, what skills and experiences participants can expect to gain, the application deadline, and an email contact for any questions.
Reach out to communities traditionally underserved in your field (e.g. community colleges, minority working groups within your professional societies)
To make your hackweek announcements more eye-catching, include photos from previous/other hackweeks that encapsulate what a hackweek is. Photos of group work and instructors teaching software programs work wonderfully. Aim to keep announcements brief, while linking to your website that will include the application form and more detailed information. Additional info that can be included in the announcement include positive testimonials from past participants.
In announcement emails, provide info on what a hackweek is, what your hackweek aims are, what skills and experiences participants can expect to gain, the application deadline, and an email contact for any questions.

Participant Selection

Important note: This is one of the most important parts of preparing the workshop, so it deserves ample time and consideration. Be aware that there are systematic biases likely present in every part of this! E.g. participants from certain backgrounds might have had fewer opportunities to learn certain skills (like coding), so requiring a certain amount of coding skills might disadvantage certain groups inadvertently! If you require things like motivational statements and/or letters of reference, be aware of biases in recommendation letters, and things like biases related to English language ability. Fundamentally, be aware that there might be biases in the process, and work to mitigate those biases wherever you can!

Any information you need for your selection process must be assessed during the application stage. This might seem obvious, but there’s always a question we forget to include.

Have a detailed conversation about what the ideal mix of participants is:

Are there prerequisites for attending the workshop (e.g. a minimum level of coding ability)?
Are there particular groups that are targeted by the workshop?

What are possible axes of selection (e.g. motivation to attend, career stage, geographic origin, skills in a certain subject, demographic variables like gender identity, race, ethnicity, etc.)? Should there be a selection for merit?
Think about ways to assess the categories of selection carefully. Any category for selection needs to be translated into some question or application material that can be provided by participants (e.g. CV? Statement of purpose? Letters of recommendation?)

What are local rules and laws (university/country) around asking about demographic variables? (note that asking participants to disclose personal demographic information should always be voluntary!)
Asking participants to self-assess skill levels is difficult and fraught with biases. If you ask people whether they’re an “expert”, “intermediate” or “beginner” at a certain skill, it’s hard to figure out what “expert” means for any given person. Try to tie categories to explicit milestones (e.g. “I have edited a python script written by others”, “I have written my own Python script”, “I maintain several open-source packages”)

Set up an application form: need to figure out what the best way to collect data is (e.g. a google form); be aware if there are local/institutional rules about data collection
Open form + advertise workshop + application (~10 months ahead of time, leave open for at least 8 weeks!)

During this time, make sure that the registration procedure/form is set up and that funding is in place: **funding must be in place by the time participants are being accepted!**
Make sure there is a back-up (ideally several back-ups) for your collected participant data. If you lose it, it’s gone and you’re in trouble!

Build infrastructure for selecting participants and decide on decision procedures:

Ask committee members to rate applicants? Need to set up categories and scoring rubrics for grading + decision metrics for selection!
Use an algorithm? Here’s one you could use: https://github.com/dhuppenkothen/entrofy
Combine both?

Run participant selection (ideally ~6 months before the workshop, to give participants ample time to make travel plans and apply for visas)
Notify participants of selection, give them a timeline for responding (especially if the workshop is oversubscribed, you might want to give them ~2 weeks so that you can rapidly admit alternative participants for those that drop out)
Iterate as participants decline/accept + run registration for those that accept
Assign funding, make sure the reimbursement procedure is clear (are people reimbursed and how? Or is the venue booking for the participants?), and inform participants how their travel/childcare expenses will be reimbursed

6 months before the workshop

Code of Conduct: As organizing committee, decide what acceptable/inacceptable behaviour looks like for your workshop, and how you can work towards being welcoming to all participants (including shy participants, participants from underrepresented groups), and settle on a Code of Conduct. Note: it is also possible to use part of the time for participants to co-create a code of conduct for the group, but this needs to be carefully facilitated.
Find and prep tutorial speakers (if you run tutorials). We know by experience that it is not straightforward even for good teachers to prepare a tutorial for a hack week. Be sure to give tutorials speakers guidance on what you would like them to talk about, and what the appropriate level is (lower than they might expect!)
Build a facilitation plan. Group activities (how participants get to know each other, how they pitch hacks, figure out groups and then work in those groups, how they work together during tutorials) work better when actively facilitated, i.e. when there are helpful structures imposed or suggested by the organizers. This is especially important to allow shy participants or more junior researchers participate fully. Work out this plan in advance. This is also your chance to experiment with new ways for people to interact/work with each other!

3-0 months before the workshop

Set up a live stream (if one is to be run). Depending on the venue, this can be either trivially easy or nearly impossible. If you are heavily oversubscribed and intend to include outside researchers via a live stream, be sure to figure out the details early.
Settle on modes of communication. How do you want participants to communicate with each other? Is slack appropriate? Set up the infrastructure for the tutorials, break-outs, hacking, and other activities you want to run

Tutorials: you could host them on GitHub and ask tutorial teachers to upload tutorials there (do this in advance so they don’t host it somewhere else!)
Hack week products: make a public folder where participants can dump things produced at the hack week (though also encourage them to upload to e.g. GitHub)
Hack idea doc: it’s useful to have a central doc where participants can add ideas for a hack and ideally discuss them in comments
Wrap-up slides: make a shared slideshow where participants can add their wrapup-slides
Set up ways for participants to communicate and coordinate online (and for outside researchers to connect): e.g. Slack
Twitter hashtag?

Potentially set up a shared software environment for participants. If the hack week tutorials involve a lot of coding in one language, it can be helpful to set up a shared environment (e.g. vial JupyterHub or Docker) for all participants to use, in order to minimize installation issues.
Send welcome e-mail to participants with important information about the workshop, the Code of Conduct, travel arrangements etc. It is also useful to encourage them at this point to start adding ideas for hacks and break-outs in the respective documents. After creating an email, designate the logistics coordinator to send out and answer all communications needed through it. They can answer general questions and forward more complex/specific inquiries to appropriate members of the team. Also, set a deadline for accepted applicants to move over to the main messaging platform that you will be using during your hackweek. One way to get all participants to join the messaging platform in a timely matter is by giving them a deadline to message the logistics coordinator their github username to be given access to needed hackweek materials.
Set up a basic evaluation survey. Set up questions on parts of the hack week you want to receive feedback on. Ideally do this before the hack week, otherwise this might cause some stress during the week itself!
The logistics coordinator should prepare check-in materials for the hackweek welcome table. This can include a roster of all participants and instructors, nametags, media release forms (if taking photos to be used for any purpose afterwards) and means for purchasing food if your hackweek is supplying any meals. Designate one coordinator to act as the main point-of-contact for all logistical questions concerning the event like meal cards, overseeing any catering and coffee set up, contacting the venue for any technological/access issues, providing assistance with figuring out travel, etc. This person should also develop a schedule of the entire event for keeping track of logistical needs at different times of throughout the event and how they will be taken care of. This will include transporting daily or time-specific supplies to and from the event. This person can also assess the venue and fill in any needs that are missing such as audio equipment, replacement batteries for microphones, and office supplies.

During the Hack Week

Important note: do not try to hack and facilitate at the same time. 1-2 organizers should be designated facilitators, and their main job should be to keep the hack week running. Trying to both hack and facilitate is very, very difficult and likely not go well.

Facilitate, facilitate, facilitate. One of the main jobs during the hack week is to make sure everyone is working in the mode that works best for them. Make sure that participants feel comfortable approaching others about contributing to their hacks, that the groups are working well, that there are no interpersonal problems.
Keep in touch with tutorial speakers: make sure they’re prepared and have all they need
Ask participants to acknowledge the hack week in publications, software, etc. This can be done for example in the acknowledgment section of a paper, or with a hashtag on GitHub.
Set up a break-out board and keep track of which break-outs have a lot of votes. You also need to find people to teach; be mindful of participants’ time and don’t overburden few of your more experienced participants with too many break-outs
Encourage experienced participants to help with the tutorial of the day, to keep them engaged with material they already know
Run regular check-in session (pitches, wrap-ups)
Be aware of impostor phenomenon being present at your event, and take steps to mitigate it

Careful facilitation can help
High representation of otherwise underrepresented groups may make minority participants less like an outsider and help mitigate impostor phenomenon
Stating its existence and prevalence in the beginning can help participants feel less alone with these emotions

Run the survey at the end of the week. It is super helpful to run the survey on the last day, because participation drops very sharply once participants have left. Set aside a little bit of time for participants to fill out the survey.
Keep notes on experiences and observations during the week. What seems to work well? What didn’t? What was missing? What could you try next time?

After the Hack Week

Evaluation

Evaluate the results of the survey. Take the survey results and find out what participants actually said about the hack week. Distill some first ideas and conclusions for next time.
Debrief. Set a meeting with the organizing committee to debrief after the hack week (ideally in the following two weeks, such that organizers had some time to think about it, but also not so far that they forget). Go through their debrief notes and the survey results, and note down changes to be made or experiments to be run the next year.
Keep track of output (papers, software, websites, news articles, blog posts, …). This is useful information to keep track of one axis of usefulness of the hack week, and also useful for future funding applications and reports

Report

Write a wrap-up report. Check whether any of your funding sources require a report, and write one if they do using the information gathered during this evaluation stage

Committee

Start choosing the committee for the next year.