Online Compilers' inner workings
If you have ever used Cyclone's Online Compiler or any other online compiler, you might have wondered how they work? How do they take your code, compile it, and run it, all in the browser? In this blog post, we will explore the inner workings of online compilers.
How do Online Compilers work?
Online Compilers are a combination of a few technologies that work together to provide you with a seamless experience of writing, compiling, and running code in the browser. Here's a high-level overview of how they work:
-
Frontend: The frontend is where you write your code. It's a text editor that allows you to write code in various languages. It also provides features like syntax highlighting, auto-completion, and error checking.
-
Backend: The backend is where the magic happens. It takes the code you've written in the frontend, compiles it using a compiler, and runs it in a sandboxed environment.
-
Communication: The frontend communicates with the backend using HTTP requests. When you click the "Run" button, the frontend sends the code to the backend, which compiles and runs it. The backend then sends the output back to the frontend, which displays it to you.
Cyclone's Online Compiler
Frontend
Cyclone's Online Compiler frontend is built using React. It provides a text editor where you can write Cyclone code. When you click the "Run" button, the frontend sends the code to the backend using an HTTP request.
Example of frontend code:
const [input, setInput] = useState("")
const runInterpreter = async () => {
if (input.match("input()")) {
alert(
"Online Compiler doesn't support Input. If you want to try out Input, download offline Compiler."
)
} else {
setOutput("")
setError("")
setLoading(true)
try {
const response = await fetch("/api/run-interpreter", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ input }),
})
if (!response.ok) {
const errorData = await response.json() // Ensure response is parsed
setError(errorData.error || "Unknown error occurred.")
return
}
const data = await response.json()
setOutput(data.output) // Insert HTML into output
setError(data.errors)
} catch (err) {
setError(err.message)
} finally {
setLoading(false)
}
}
}
return (
<div>
<Editor
value={input}
onChange={(value) => setInput(value)}
placeholder="Write your code here..."
/>
<Button onClick={runInterpreter}>Run</Button>
</div>
)Backend
Cyclone's Online Compiler backend is built using Node.js. It uses the Cyclone Interpreter to compile and run the code. The backend receives the code from the frontend, compiles it using the interpreter, and sends the output back to the frontend.
Steps in the backend code:
Step 1: Receive the code from the frontend
Convert the request body to JSON and check if the input content is present.
export async function POST(req: Request): Promise<Response> {
const { input } = await req.json()
if (!input) {
return new Response(
JSON.stringify({ error: "Input content is required." }),
{ status: 400 }
)
}
const { output, errors } = await runInterpreter(input)
}Step 2: Compile and run the code
Using Node.js's child_process.spawn method, spawn the interpreter and write the input to its stdin. Read the output and errors from the interpreter's stdout and stderr streams.
async function runInterpreter(
input: string
): Promise<{ output: string, errors: string }> {
const interpreter = spawn("interpreter", [], { shell: true })
interpreter.stdin.write(input)
interpreter.stdin.end()
let output = ""
let errors = ""
for await (const data of interpreter.stdout) {
output += data
}
for await (const data of interpreter.stderr) {
errors += data
}
return { output, errors }
}Step 3: Sanitize the output
Remove ANSI colors from the output and replace newlines with <br /> tags.
const removeAnsiColors = (str: string): string => {
const ansiColorMap: { [key: string]: string } = {
"\x1b[31m": "color: red;",
"\x1b[32m": "color: green;",
"\x1b[33m": "color: yellow;",
"\x1b[34m": "color: blue;",
"\x1b[35m": "color: magenta;",
"\x1b[36m": "color: cyan;",
"\x1b[90m": "color: gray;",
"\x1b[0m": "",
}
const htmlWithColor = str
.replace(/\x1b\[[0-9;]*m/g, (match) => {
if (ansiColorMap[match]) {
return `<span style="${ansiColorMap[match]}">`
}
return ""
})
.replace(/\x1b\[0m/g, "</span>")
return htmlWithColor.replace(/\n/g, "<br />")
}
output = removeAnsiColors(output)Step 4: Send the output back to the frontend
Send the output and errors back to the frontend as JSON.
return new Response(
JSON.stringify({ output, errors }),
{ headers: { "Content-Type": "application/json" } }
)Step 5: Display the output in the frontend
Display the output and errors in the frontend.
<div>
<div dangerouslySetInnerHTML={{ __html: output }} />
{errors && <div className="error">{errors}</div>}
</div>Bonus Step: Handle Infinite Loops
To prevent infinite loops, you can use a timeout to kill the interpreter process after a certain time. Also turncate the output if it exceeds a certain length.
const timeout = setTimeout(() => {
console.warn("Timeout reached. Killing process.")
process.kill("SIGKILL") // Forcefully terminate the process
}, 1000)Final Code:
import fs from 'fs';
import path from 'path';
import { spawn } from 'child_process';
const removeAnsiColors = (str: string): string => {
const ansiColorMap: { [key: string]: string } = {
'\x1b[31m': 'color: red;',
'\x1b[32m': 'color: green;',
'\x1b[33m': 'color: yellow;',
'\x1b[34m': 'color: blue;',
'\x1b[35m': 'color: magenta;',
'\x1b[36m': 'color: cyan;',
'\x1b[90m': 'color: gray;',
'\x1b[0m': ''
};
const htmlWithColor = str.replace(/\x1b\[[0-9;]*m/g, (match) => {
if (ansiColorMap[match]) {
return `<span style="${ansiColorMap[match]}">`;
}
return '';
}).replace(/\x1b\[0m/g, '</span>');
return htmlWithColor.replace(/\n/g, '<br />');
};
export async function POST(req: Request): Promise<Response> {
const { input } = await req.json();
if (!input) {
return new Response(JSON.stringify({ error: 'Input content is required.' }), { status: 400 });
}
try {
const interpreterPath = path.join(process.cwd(), 'binaries', 'cyinterpreter'); // Path to the interpreter binary
const inputFilePath = path.join('/tmp', 'input.cy');
const tempDir = path.dirname(inputFilePath);
if (!fs.existsSync(tempDir)) {
fs.mkdirSync(tempDir, { recursive: true });
console.log("Created temp directory");
}
console.log("Writing input file.");
fs.writeFileSync(inputFilePath, input, 'utf-8');
console.log("Done writing input file.");
console.log("Interpreter path:", interpreterPath);
console.log("Does interpreter exist?:", fs.existsSync(interpreterPath));
return new Promise<Response>((resolve, reject) => {
console.log("Running interpreter.");
const timeoutMessage = "Execution exceeded timeout of 1000ms, possibly infinite loop. Displaying first 256 characters of output:<br />";
let stdoutData = "";
let stderrData = "";
const maxOutputLength = 256;
const process = spawn(interpreterPath, [inputFilePath]);
const timeout = setTimeout(() => {
console.warn("Timeout reached. Killing process.");
process.kill('SIGKILL'); // Forcefully terminate the process
}, 1000);
process.stdout.on("data", (data) => {
if (stdoutData.length < maxOutputLength) {
stdoutData += data.toString();
if (stdoutData.length > maxOutputLength) {
stdoutData = stdoutData.slice(0, maxOutputLength); // Truncate to maximum length
stdoutData += '...'; // Indicate truncation
}
}
});
process.stderr.on("data", (data) => {
stderrData += data;
});
process.on("close", (code) => {
clearTimeout(timeout); // Clear the timeout if the process exits normally
const cleanedStdout = removeAnsiColors(stdoutData);
const responseMessage = code === null
? timeoutMessage + cleanedStdout
: cleanedStdout;
resolve(new Response(JSON.stringify({ output: responseMessage }), { status: 200 }));
});
process.on("error", (error) => {
clearTimeout(timeout); // Clear the timeout on error
reject(new Response(JSON.stringify({ error: 'Execution failed', details: error.message }), { status: 500 }));
});
});
} catch (err: any) {
return new Response(JSON.stringify({ error: 'Server error', details: err.message }), { status: 500 });
}
}
Conclusion
Online compilers are a great way to quickly test your code without setting up a development environment. They use a combination of frontend and backend technologies to provide you with a seamless coding experience. Understanding how they work can help you appreciate the effort that goes into building them and give you insights into how you can build your own.
Hope you enjoyed this blog post. If you have any questions or feedback, feel free to reach out to us.
Happy coding!
Jayvardhan Patil
DevTomorrow