Autograder-Safe Code: A Simple Checklist for Students

student is checking all possible grader erros before submitting the assignment.

You finished your programming homework, tested it, and everything works correctly on your system. You submit to the autograder, and to your surprise, you only see red crosses: Time Limit Error, Memory Limit Error, Runtime Error, or worse, your code doesn’t even compile.

Just because your program works on your laptop doesn’t mean it will work everywhere. Autograders are strict; they test your submission on edge cases, on large inputs, on efficiency, on performance, on memory correctness, and even on coding style.

This tutorial provides a practical checklist to make your code safe for the autograder. From compiling with strict flags and ensuring portability to catching undefined behavior, optimizing input and output, and preventing memory issues, we’ll go through the most common reasons submissions fail and how to avoid them.

Checklist for Students

  1. Compile with strict flags (-Wall -Wextra -Werror -pedantic).
  2. Compile with ISO standard for maximum portability. (-std=c90, -std=c++11).
  3. Test with optimizations enabled (e.g., -O2).
  4. Run a static analyzer (e.g., clang-tidy, cppcheck) to detect undefined behavior and errors.
  5. Test with Valgrind to check for memory leaks or errors.
  6. In C++ disable synchronization and untie cin from cout for fast I/O.
  7. Do not use std::endl, it causes stdout to flush, use ‘\n’ instead to print a new line.
  8. Batch output to prevent TLE.
  9. Don’t mix signed and unsigned arithmetic to prevent overflows and undefined behavior.
  10. Avoid using string functions with binary data. For example: use memcpy instead of strcpy.
  11. Check all system call return values (e.g., malloc, read)
  12. Verify the program exit code after program execution. 0 for success and non-zero for failures.
  13. Unix uses \n (LF) and Windows uses \r\n (CRLF). Always submit with LF unless stated otherwise.
  14. Guard against division/modulo by zero.
  15. Avoid large allocations on stack. Use heap memory (malloc, new) instead.
  16. Beware of recursion depth errors on large inputs.
  17. Use relative paths (./data/input.txt) instead of (C:\Users…).
  18. Keep consistent code style.
  19. Remove binaries, object files, log files before packaging your submission.

Add Your Heading Text Here

Do not rely on default compilation like:

gcc main.c

Autograder compiles your code with stricter compiler flags settings different from your machine. Code that compiles locally may fail to compile with the autograder. Before submission, always compile your code with strict flags.

gcc -Wall -Wextra -Werror -pedantic -std=c90

This enforces ISO C90 standard and turns on all warnings and treats them as errors.
You can use other ISO standards like -std=c99, -std=c11 if your assignment allows you to use modern C features.
In case your assignment allows you to use POSIX API like open, read, write, socket, etc, use gnu standards like std=gnu90.

Rule of thumb:
1- Use c89/c90 for maximum portability
2- Use gnu standards only if the assignment requires you to use POSIX API.

Check Code Under Optimization Flags

Your program may have undefined behaviour such as uninitialized variables which may run fine locally, but under optimizations can expose them by crashing the program or producing unexpected results.

gcc -Wall -Wextra -Werror -pedantic -std=c90 -O2

This enables optimization level 2 which is standard and commonly used by autograders. You can also try -O3 for more aggressive optimizations.

Run a Static Analyzer

Many autograder failures come from undefined behavior. Before submission, run a static analyser tool on your code. These tools help you to detect memory issues, bug-prone idioms, issues like integer overflows, null pointer dereferences, out of bounds access, uninitialized variables, etc,. all without running your program. Popular C/C++ tools are cppcheck, clang-tidy and pylint for python.
C/C++

cppcheck –enable=all main.cpp
cppcheck –enable=all <path_to_folder>

–enable=all: turns on all the checks like style, warnings, performance, portability, etc.

⚠️ cppcheck often struggles with standard include headers, on Mac and Linux, you can suppress this check by:

cppcheck –enable=all –suppress=missingIncludeSystem main.cpp

source
Python
you can use pylint for python programspylint main.py

Check for memory leaks or errors using valgrind tool

Autograder will run your code under valgrind to check that it’s free from leaks or runtime errors. Even if your program produces the correct answer, failing valgrind means a failed submission or deduction of points. Valgrind can catch mistakes like forgetting to free memory or resource leaks like leaving sockets or file descriptors open. It also reports memory errors which don’t get caught during normal program execution. If valgrind runs cleanly locally, it is much more likely to run cleanly on autograder systems.

Before you can use valgrind, you need to compile the code with a debug flag using the -g option. -g enables the compiler to attach debug symbols in your executable so valgrind can show line numbers where the leak or an error occurred.

gcc -g main.c

Run the following valgrind command to test for memory errors and leaks.

valgrind –trace-children=yes –track-fds=yes –track-origins=yes –leak-check=full –show-leak-kinds=all ./a.out

The following is a valgrind output showing no leaks and errors.


==20586== FILE DESCRIPTORS: 3 open (3 std) at exit.
==20586== HEAP SUMMARY:
==20586== in use at exit: 0 bytes in 0 blocks
==20586== total heap usage: 1 allocs, 1 frees, 73,728 bytes allocated
==20586== All heap blocks were freed — no leaks are possible
==20586== For lists of detected and suppressed errors, rerun with: -s
==20586== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)


Memory leak in system libraries.
There might be leaks in the system libraries like printf which is not your fault. passing –suppressions flag
suppresses leaks from them.

Timeout due to Stdin / Stdout

Background on buffer and flush
Let’s understand what a buffer is. scanf/printf and cin/cout are built on top of read() and write() system call.
To avoid calling these system calls frequently they maintain their own internal buffer.
Input (scanf/cin)
When you read input, the library makes a single call to read() system call to read a large chunk of data and store in its internal
input buffer. Subsequent calls to scanf and cin then read from this buffer. If the buffer runs out of data, another read() is invoked.
Output (printf/cout)
When you print, the data is stored in its internal output buffer. Once the buffer is filled or when a flush is requested, the buffer contents
are written out using the write system call(). Frequent flush operations can become expensive due to invocation of write().
when flushing occurs is determined by buffering mode:
Line buffered: the output is flushed when a new line is printed.
Fully Buffered: the output is flushed when the internal buffer is completely filled or program ends or manually flushed. This is the default mode when stdout is redirected to a file and autograders usually do redirect.
Unbuffered: the output is immediately written.

Now let’s understand why your program might be causing TLE.
Slow cin/cout
In C++, cout and cin are synchronized with printf and scanf. Additionally, cin is tied to cout, which results in cout being flushed before each input operation. Both of these reasons slow down the performance of cout/cin. To speed up their performance you need to disable synchronization and untie cin from cout.

std::ios::sync_with_stdio(false); // disable sync with stdio
std::cin.tie(nullptr); // untie cin from cout

Flush due to std::endl
Using std::endl implicitly flushes cout which may cause performance issues if you’re using too much for example in a loop. Use ‘\n’ instead to print a new line. Let’s compare the timings of std::endl and ‘\n’.
Using std:endl to print a new line

// slow
for (int i = 0; i < 100000000; i++) {
std::cout << output << std::endl;
}

$ time ./a.out > out
real 0m23.017s
user 0m6.405s
sys 0m16.591s

Using ‘\n’ to print a new line

// fast
for (int i = 0; i < 100000000; i++) {
std::cout << output << '\n';
}

$ time ./a.out > out
real 0m1.808s
user 0m1.575s
sys 0m0.233s

Replacing std::endl with '\n' sped up the program by ~12.7x.
Batch your output
If you are printing too much for example inside a very large loop, they can slow down the performance of the program due to internal flushing. Instead you can output inside your own buffer and perform a single output after the loop or when the buffer is full.
Batching example using C

// C fast, uses a custom buffer to collect output and
// prints when the buffer is full or finally at the end.
char buffer[MAX_BUFFER_LENGTH];
size_t buffer_length = 0;
for (int i = 0; i < 10000000; i++) {
int length = sprintf(buffer + buffer_length, "%d: hello\n", i);
buffer_length += length;
if (buffer_length >= FLUSH_LIMIT) {
buffer[buffer_length] = '\0';
printf("%s", buffer);
buffer_length = 0;
}
}

// edge case, what if the buffer_length never crossed FLUSH_LIMIT ?
if (buffer_length != 0) {
buffer[buffer_length] = '\0';
printf("%s", buffer);
}

Here we are assuming there is a good amount of difference between MAX_BUFFER_LENGTH and FLUSH_LIMIT such that sprintf never writes past MAX_BUFFER_LENGTH. Use safe snprintf instead of unsafe sprintf to prevent buffer overflow errors.
Batching example using C++

ostringstream out;
for (int i = 0; i < 10000000; i++) {
out << i << ": hello\n";
}
cout << out.str();

Do not use string functions with binary data.

Bugs can be caused due to using string functions like strcpy, strlen, strcmp on binary data.
Your program will look logically correct and pass the sample test cases but on final judging, the autograder can pass \0, a null character and string functions stop processing after reading them which fails your submission. Also while local testing you cannot type null byte from stdin and you can miss this. Always use mem* functions like memcpy, memmove, memcmp on binary data.

char src[5] = { ‘c’, ‘o’, ‘\0’, ‘d’, ‘e’ };
char dst[5];
strcpy(dst, src); // stops at ‘\0’, copies only “co”
memcpy(dst, src, 5); // copies all the 5 bytes

Do not create large data on stack.

Beware of creating large arrays in your functions. Locally, the stack might be bigger and the program will run fine. But autograder might impose strict limits on the stack size and the same code will crash with a segmentation fault. Avoid putting large data on the stack, use malloc instead.
Lets test this idea
Below is a program which takes a stack size in MB as a command line argument and creates a 1 MB array on the stack. The program works fine if the stack is greater than 1 MB otherwise, it crashes.


#include <sys/resource.h>
#include <stdio.h>
#include <stdlib.h>

void test()
{
char arr[1024 * 1024]; // create 1mb of data on stack
}


int main(int argc, char **argv)
{
// set the stack space
if (argc > 1)
{
unsigned long long mb = atoi(argv[1]) * 1024ULL * 1024ULL;
struct rlimit rl = {.rlim_cur = mb, .rlim_max = mb};
if (setrlimit(RLIMIT_STACK, &rl) != 0)
{
perror("setrlimit");
return 1;
}
printf("Stack size set to %llu bytes\n", mb);
}
else
{
printf("Stack size not passed, set to default\n");
}


printf("Creating 1mb of data\n");
fflush(stdout);
test();


return 0;
}

$ gcc main.c
$
$ ./a.out 2
Stack size set to 2097152 bytes
Creating 1mb of data
$
$ ./a.out 1
Stack size set to 1048576 bytes
Creating 1mb of data
Segmentation fault (core dumped)

Division / Modulo By Zero

Most of the time we write code like this int z = x / y. Here we might miss the fact that y might be zero and is undefined behaviour in C and will terminate with exception in other languages like python / java.
Shockingly, due to undefined behaviour, sometimes the program may seem to work fine locally and you might miss this bug and will crash on autograder. Same is the case with modulus operation.

Always guard against this edge case by writing code like this:

int z = 0;
int w = 0;
if (y != 0) {
z = x / y;
w = x % y;
}

Do Not Mix Signed and Unsigned Arithmetic

Use unsigned types cautionally. Even the Google C++ Style Guide admits:
Using unsigned integers for container sizes was a historical mistake. Unsigned arithmetic hides bugs and prevents useful compiler warnings. Unsigned integers are good for representing bitfields and modular arithmetic.

STL containers return size_t for size(), and can risk wrapping bugs.

vector <int> arr;
for (size_t i = 0; i < arr.size() - 1; i++);

This looks innocent, but arr.size() returns an unsigned type and when the size is 0, subtracting -1 will give you a MAX number instead of -1 and will loop for a long time.

Check Exit Code

Use echo $? to check the exit code of your program. Make sure it’s 0. Non-zero exit code indicates errors or failure. The autograder may interpret any non-zero values as failure even when the program produces the correct output.

$ ./a.out 2
Stack size set to 2097152 bytes
Creating 1mb of data
$ echo $?
0

$ ./a.out 1
Stack size set to 1048576 bytes
Creating 1mb of data
Segmentation fault (core dumped)
$ echo $?
139

Check Return Value from System Calls.

Always verify the return value from system calls. For example, the malloc() call will succeed locally on your machine but might fail on the autograder due to stricter resource limits. You will be frustrated why your code is crashing on the autograder even if it’s logically correct. Exit the program cleanly in case of an error and use perror() to print the reason for failure.

// Allocated 1 GB
void* buffer = malloc(1ULL * 1024 * 1024 * 1024);
if (buffer == NULL) {
perror("malloc failed");
exit(EXIT_FAILURE);
}

Don’t Hardcode Paths

Autograder will extract your code in a sandboxed environment that has a different directory structure than your local machine. If you hardcode paths such as “/home/john/assignment/data/file.txt”, your program will not be able to open it on the Autograder since that path is not present in its environment. Instead, always use relative paths, for example “./data/file.txt”.

There are few additional pitfalls. Linux paths are case-sensitive, so “./data/file.txt” is not the same as “./Data/file.txt”. This will work on Windows where the paths are case-insensitive but fails on Linux, and most autograders run on Linux. Another common issue is path separators. Unix uses forward slash / as a path separator while Windows uses backward slash \ as well as forward slash / for C API like fopen. If you write your code in a windows environment make sure you use forward slashes / to be more portable.

Line Terminators

Each platform has their own way of terminating a line. The Unix system uses ‘\n’, Windows uses \r\n.
Suppose your professor gives you a text file for your assignment, you would usually expect \n as a newline character. If the text file was created on Windows, the newline is marked by \r\n instead. As a result, you may read \r as part of your data and use \n to identify a new line. This extra \r can be the cause of unexpected bugs. To avoid this, use tools like dos2unix to convert Windows text files to unix based text files where \r\n is replaced with \n. We have faced this bug multiple times. In case of more foolproof, while you code expect that autograder might feed you \r\n as a line terminator.

Use Consistent Code Styling

Autograder checks for code style and there are penalties

1- for inconsistent styles.
2- Use consistent bracing style.
3- Use a single variable naming convention (eg., snake_case or camelCase)
4- Use spaces for indentation style, don’t mix tabs and spaces.
5- Keep line length to ~80 characters for readability
6- Avoid Magic numbers, use names constants instead.
7- Write contents in ALL_CAPS.
8- Use tools like clang-format for formatting C/C++ code and black/autopep8 for formatting python code.

Packaging & Submission

Before packaging your code for submission make sure to delete any files and folders not part of your assignment. Remove hidden folders like .idea, .vscode, object files, log files, executables. If your project uses a Makefile, always run make clean before packaging. Also include a README file to help the instructors know how to compile your program and any other information you want the instructor to know.
Double-check the required directory structure. Don’t wrap your project inside an extra top-level folder unless explicitly told.

Conclusion

Autograder runs your program under a strict environment and tests with various edge cases. To write autograder-safe code, you need to write your program for maximum portability, check for undefined behavior with a static analyzer, and always test your program with optimizations enabled. Use tools like Valgrind to detect any memory leaks and runtime errors. Stress test your program with a wide range of edge cases and invalid inputs. Check your program’s efficiency on large inputs. By following the checklist, you can eliminate the most common problems that cause your submission to fail. 

After making all the attempts, still if you feel you need an extra hand, then you can ask for coding homework help from our expert, and you will get the best possible assistance. from MyCodingpal’s coding experts.

FAQ's

Should I flush stdout manually?

Stdout is automatically flushed when the program ends. However, in interactive programs where the next input depends on the content of the output, you need to flush stdout before each input operation.

Why does Windows CRLF break my grader?

Windows uses \r\n (CRLF) as the line terminator and Linux uses ‘\n’ (LF). Most autograders run on Linux and compare output byte by byte. This extra \r can fail the submission. Always print newlines using \n. Additionally, If you’re submitting any text files created on Windows as part of the assignment solution, make sure the lines are ended by ‘\n’. You can use the dos2unix tool to convert CRLF to LF before submission.

My program behaves unexpectedly with the sample test case files. What should I do?

If the test case files were created on Windows, they may contain \r\n instead of \n for new lines. Run the sample testcase files under dos2unix tool to convert \r\n to \n.

Do I need a newline at the end of the output?

Autograders compare output byte by byte and if the expected output ends with a newline and your output doesn’t, the submission will fail. Also Unix text files always end with a newline by convention. So ending with newline avoids mismatch.

Why does my program get a segmentation fault on the autograder but not on my machine?

There might be undefined behaviour in your code. Run your program under the valgrind tool, test your program with optimization flag -O2. Also run a static analyzer on your code to detect possible errors.

Why does my program get TLE (Time Limit Exceeded)?

Your program might be getting TLE on large inputs due to use of inefficient algorithms or data structure.

Why does my program get RE (Runtime Error)?

Runtime errors can be caused by division or modulo by zero, dereferencing null pointers, accessing arrays at invalid indices, maybe your buffer is too small and overflowing. Allocating large arrays on the stack can also cause runtime errors. Not checking the return values of library or system calls, maybe the malloc call is returning NULL. On large inputs runtime errors can be due to allocating too much memory or your recursive function is reaching depth limits. To catch these issues, run your code under Valgrind and stress test on large inputs and edge cases and use sanitizers (-fsanitize=address,undefined).

Can I print debug statements during submission?

Do not print debug messages to stdout. The autograder performs a byte-by-byte comparison and your submission will fail. Do not print anything other than what is required.

Why does my program fail only with large inputs?

Autograder feed inputs in order of 10^7 to check how efficient your program is. Common issues include using algorithms with poor time complexity, slower data structures, slow IO, maybe you’re creating too much memory, recursion depth limits are common issues with python, integer overflows due to int, unnecessary copies, using std::endl instead of ‘\n’ to print new lines, print too often in hot loop.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top