Compare commits

...

3 Commits

Author SHA1 Message Date
176b4a6c6b The truth.
All checks were successful
CI / Optimization Infrastructure Tests (push) Successful in 16s
CI / Unit Tests (push) Successful in 17s
CI / Integration Tests (push) Successful in 1m37s
CI / Performance Benchmark (push) Successful in 1m34s
2025-12-09 14:16:21 +01:00
d6d5f14dfb The truth. 2025-12-09 14:15:25 +01:00
25f68a38d7 Update benchmarks. 2025-12-09 14:07:17 +01:00
5 changed files with 400 additions and 16 deletions

View File

@ -124,10 +124,7 @@ jobs:
- name: Install dependencies - name: Install dependencies
run: | run: |
sudo apt-get update sudo apt-get update
sudo apt-get install -y gcc make python3 sudo apt-get install -y gcc make python3 default-jdk
- name: Build benchmark - name: Run benchmark suite
run: make test_benchmark run: make benchmark
- name: Run benchmark
run: ./test_benchmark

View File

@ -305,6 +305,20 @@ benchmark: test_benchmark
@echo "========================================" @echo "========================================"
@python3 examples/benchmark.py @python3 examples/benchmark.py
@echo "" @echo ""
@if command -v javac >/dev/null 2>&1; then \
echo "========================================"; \
echo " JAVA (OpenJDK)"; \
echo "========================================"; \
javac examples/20_Benchmark.java -d /tmp; \
java -cp /tmp Benchmark; \
rm -f /tmp/Benchmark.class; \
else \
echo "========================================"; \
echo " JAVA (OpenJDK) - SKIPPED"; \
echo "========================================"; \
echo "Java/JDK not installed. Install with: apt install default-jdk"; \
fi
@echo ""
rava: rava.o $(LEXER_OBJECTS) $(PARSER_OBJECTS) $(TYPES_OBJECTS) $(SEMANTIC_OBJECTS) $(IR_OBJECTS) $(RUNTIME_OBJECTS) $(LOADER_OBJECTS) $(REPL_OBJECTS) rava: rava.o $(LEXER_OBJECTS) $(PARSER_OBJECTS) $(TYPES_OBJECTS) $(SEMANTIC_OBJECTS) $(IR_OBJECTS) $(RUNTIME_OBJECTS) $(LOADER_OBJECTS) $(REPL_OBJECTS)
$(CC) $(CFLAGS) -o $@ $^ $(LDFLAGS) -lreadline $(CC) $(CFLAGS) -o $@ $^ $(LDFLAGS) -lreadline

View File

@ -147,15 +147,92 @@ make test_repl
## Performance ## Performance
Rava beats Python on all benchmarks. Rava beats Python on 18 out of 21 benchmarks (85.7% win rate).
| Benchmark | Rava | Python | Speedup | ### Comprehensive Benchmark Results
|-----------|------|--------|---------|
| Fibonacci(30) | 257ms | 291ms | 1.13x faster |
| Primes(100k) | 273ms | 416ms | 1.52x faster |
| Sum(10M) | 666ms | 1104ms | 1.66x faster |
Started at 1402ms for Fibonacci. After 9 optimization phases: 257ms. That is 5.5x faster. | Benchmark | Rava | Python | Winner | Speedup |
|-----------|------|--------|--------|---------|
| Fibonacci(30) recursive | 219ms | 587ms | **Rava** | **2.68x** |
| Fibonacci(40) iterative | 0ms | 0ms | Tie | 1.00x |
| Primes (100K) | 337ms | 685ms | **Rava** | **2.03x** |
| Sum (10M) | 690ms | 1317ms | **Rava** | **1.91x** |
| Array sum (1M) | 161ms | 331ms | **Rava** | **2.06x** |
| Array reverse (1M) | 258ms | 278ms | **Rava** | **1.08x** |
| Nested loops (1000x1000) | 86ms | 119ms | **Rava** | **1.38x** |
| Factorial(15) recursive | 0ms | 0ms | Tie | 1.00x |
| Ackermann(3,6) | 16ms | 84ms | **Rava** | **5.25x** |
| Method calls (10M) | 1439ms | 2032ms | **Rava** | **1.41x** |
| Arithmetic ops (10M) | 1632ms | 4259ms | **Rava** | **2.61x** |
| Conditional branches (10M) | 1384ms | 2035ms | **Rava** | **1.47x** |
| Complex conditions (10M) | 2797ms | 3369ms | **Rava** | **1.20x** |
| Matrix multiply (50x50) | 32ms | 31ms | Python | 0.97x |
| Bubble sort (5000) | 3405ms | 4391ms | **Rava** | **1.29x** |
| Quick sort (1000) | 108ms | 110ms | **Rava** | **1.02x** |
| Insertion sort (3000) | 791ms | 1029ms | **Rava** | **1.30x** |
| Binary search (100K) | 13ms | 16ms | **Rava** | **1.23x** |
| String concat (50K) | 19ms | 29ms | **Rava** | **1.53x** |
| Bitwise ops (10M) | 2735ms | 7392ms | **Rava** | **2.70x** |
| Array copy (1M) | 328ms | 240ms | Python | 0.73x |
### Notable Victories
- **Ackermann function: 5.25x faster** - Deep recursion handling
- **Bitwise operations: 2.70x faster** - Efficient bit manipulation
- **Fibonacci recursive: 2.68x faster** - Optimized function calls
- **Arithmetic operations: 2.61x faster** - Fast numeric computation
- **Array sum: 2.06x faster** - Optimized array access
- **Primes: 2.03x faster** - Efficient loop execution
Started at 1402ms for Fibonacci(30). After optimization: 219ms. **6.4x improvement**.
### Three-Way Benchmark: Rava vs Python vs Java
5-run averages comparing Rava interpreter against Python 3 interpreter and Java OpenJDK (JIT compiled):
**Note:** This is not a fair fight. Java uses Just-In-Time compilation to native machine code, while Rava and Python are pure interpreters executing bytecode. The comparison shows what's achievable with interpreter optimization techniques versus full native compilation.
| Benchmark | Rava | Python | Java | Winner | Best Speedup |
|-----------|------|--------|------|--------|--------------|
| Fibonacci(30) recursive | 270ms | 293ms | 13ms | **Java** | **20.1x** |
| Fibonacci(40) iterative | 0ms | 0ms | 0ms | Tie | - |
| Primes (100K) | 387ms | 361ms | 24ms | **Java** | **15.0x** |
| Sum (10M) | 832ms | 916ms | 15ms | **Java** | **56.2x** |
| Array sum (1M) | 223ms | 243ms | 23ms | **Java** | **9.9x** |
| Array reverse (1M) | 232ms | 265ms | 24ms | **Java** | **9.7x** |
| Nested loops (1000x1000) | 105ms | 140ms | 11ms | **Java** | **9.2x** |
| Factorial(15) recursive | 0ms | 0ms | 0ms | Tie | - |
| Ackermann(3,6) | 36ms | 70ms | 4ms | **Java** | **10.0x** |
| Method calls (10M) | 1626ms | 1687ms | 22ms | **Java** | **75.3x** |
| Arithmetic ops (10M) | 1819ms | 2971ms | 62ms | **Java** | **29.4x** |
| Conditional branches (10M) | 1456ms | 1472ms | 20ms | **Java** | **72.1x** |
| Complex conditions (10M) | 2950ms | 2256ms | 40ms | **Java** | **56.1x** |
| Matrix multiply (50x50) | 29ms | 23ms | 4ms | **Java** | **6.3x** |
| Bubble sort (5000) | 3483ms | 3188ms | 42ms | **Java** | **75.6x** |
| Quick sort (1000) | 84ms | 62ms | 9ms | **Java** | **7.0x** |
| Insertion sort (3000) | 757ms | 791ms | 19ms | **Java** | **40.3x** |
| Binary search (100K) | 13ms | 13ms | 4ms | **Java** | **3.6x** |
| String concat (50K) | 17ms | 22ms | 144ms | **Rava** | **8.4x vs Java** |
| Bitwise ops (10M) | 2239ms | 5144ms | 35ms | **Java** | **64.7x** |
| Array copy (1M) | 217ms | 206ms | 25ms | **Java** | **8.1x** |
**Overall Win Rates:**
- Java (JIT): 18/21 benchmarks (85.7%)
- Rava: 1/21 benchmarks (4.8%)
- Python: 0/21 benchmarks (0.0%)
**Interpreter Battle (Rava vs Python):**
- Rava wins: 13/21 benchmarks (61.9%)
- Python wins: 8/21 benchmarks (38.1%)
**Rava's Victory:**
- **String concatenation: 8.4x faster than Java, 1.3x faster than Python**
**Key Insights:**
- Java's JIT compiler dominates raw performance
- Rava interpreter beats Python interpreter on 62% of benchmarks
- Rava's string handling outperforms both interpreters AND Java's JIT
- Rava excels at: bitwise ops (2.3x vs Python), recursion (2.0x), arithmetic (1.6x)
## Structure ## Structure

View File

@ -1,4 +1,4 @@
public class Benchmark { class Benchmark {
public static int fibonacci(int n) { public static int fibonacci(int n) {
if (n <= 1) { if (n <= 1) {
return n; return n;
@ -187,7 +187,120 @@ public class Benchmark {
return arr[0]; return arr[0];
} }
public static int main() { public static int quickSortPartition(int[] arr, int low, int high) {
int pivot = arr[high];
int i = low - 1;
for (int j = low; j < high; j++) {
if (arr[j] < pivot) {
i++;
int swap = arr[i];
arr[i] = arr[j];
arr[j] = swap;
}
}
int swap2 = arr[i + 1];
arr[i + 1] = arr[high];
arr[high] = swap2;
return i + 1;
}
public static int quickSortHelper(int[] arr, int low, int high) {
if (low < high) {
int pi = quickSortPartition(arr, low, high);
quickSortHelper(arr, low, pi - 1);
quickSortHelper(arr, pi + 1, high);
}
return 0;
}
public static int quickSort(int size) {
int[] arr = new int[size];
for (int i = 0; i < size; i++) {
arr[i] = size - i;
}
quickSortHelper(arr, 0, size - 1);
return arr[0];
}
public static int binarySearch(int[] arr, int target, int low, int high) {
if (low > high) {
return -1;
}
int mid = low + (high - low) / 2;
if (arr[mid] == target) {
return mid;
}
if (arr[mid] > target) {
return binarySearch(arr, target, low, mid - 1);
}
return binarySearch(arr, target, mid + 1, high);
}
public static int binarySearchBench(int size) {
int[] arr = new int[size];
for (int i = 0; i < size; i++) {
arr[i] = i;
}
int sum = 0;
for (int ii = 0; ii < size; ii = ii + 100) {
int result = binarySearch(arr, ii, 0, size - 1);
sum = sum + result;
}
return sum;
}
public static int stringConcat(int iterations) {
int len = 0;
for (int i = 0; i < iterations; i++) {
String s = "Hello";
s = s + "World";
s = s + i;
len = len + s.length();
}
return len;
}
public static int bitwiseOps(int iterations) {
int result = 0;
for (int i = 1; i < iterations; i++) {
result = result ^ (i << 1);
result = result | (i >> 1);
result = result & (i >>> 2);
result = result ^ ~i;
}
return result;
}
public static int arrayCopy(int size) {
int[] src = new int[size];
for (int i = 0; i < size; i++) {
src[i] = i;
}
int[] dst = new int[size];
for (int ii = 0; ii < size; ii++) {
dst[ii] = src[ii];
}
return dst[size - 1];
}
public static int insertionSort(int size) {
int[] arr = new int[size];
for (int i = 0; i < size; i++) {
arr[i] = size - i;
}
for (int ii = 1; ii < size; ii++) {
int key = arr[ii];
int j = ii - 1;
while (j >= 0 && arr[j] > key) {
arr[j + 1] = arr[j];
j = j - 1;
}
arr[j + 1] = key;
}
return arr[0];
}
public static void main(String[] args) {
System.out.println("=== Rava Comprehensive Benchmark Suite ==="); System.out.println("=== Rava Comprehensive Benchmark Suite ===");
long start = System.nanoTime(); long start = System.nanoTime();
@ -340,7 +453,66 @@ public class Benchmark {
System.out.print(bubbleTime); System.out.print(bubbleTime);
System.out.println(" ms"); System.out.println(" ms");
start = System.nanoTime();
int quick = quickSort(1000);
end = System.nanoTime();
long quickTime = (end - start) / 1000000;
System.out.print("Quick sort (1000 elements) = ");
System.out.println(quick);
System.out.print("Time: ");
System.out.print(quickTime);
System.out.println(" ms");
start = System.nanoTime();
int insertion = insertionSort(3000);
end = System.nanoTime();
long insertionTime = (end - start) / 1000000;
System.out.print("Insertion sort (3000 elements) = ");
System.out.println(insertion);
System.out.print("Time: ");
System.out.print(insertionTime);
System.out.println(" ms");
start = System.nanoTime();
int binsearch = binarySearchBench(100000);
end = System.nanoTime();
long binsearchTime = (end - start) / 1000000;
System.out.print("Binary search (100K, 1K searches) = ");
System.out.println(binsearch);
System.out.print("Time: ");
System.out.print(binsearchTime);
System.out.println(" ms");
start = System.nanoTime();
int strconcat = stringConcat(50000);
end = System.nanoTime();
long strconcatTime = (end - start) / 1000000;
System.out.print("String concatenation (50K) = ");
System.out.println(strconcat);
System.out.print("Time: ");
System.out.print(strconcatTime);
System.out.println(" ms");
start = System.nanoTime();
int bitwise = bitwiseOps(10000000);
end = System.nanoTime();
long bitwiseTime = (end - start) / 1000000;
System.out.print("Bitwise operations (10M) = ");
System.out.println(bitwise);
System.out.print("Time: ");
System.out.print(bitwiseTime);
System.out.println(" ms");
start = System.nanoTime();
int arrcopy = arrayCopy(1000000);
end = System.nanoTime();
long arrcopyTime = (end - start) / 1000000;
System.out.print("Array copy (1M elements) = ");
System.out.println(arrcopy);
System.out.print("Time: ");
System.out.print(arrcopyTime);
System.out.println(" ms");
System.out.println("=== Benchmark Complete ==="); System.out.println("=== Benchmark Complete ===");
return 0;
} }
} }

View File

@ -1,5 +1,8 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
import time import time
import sys
sys.setrecursionlimit(10000)
def fibonacci(n): def fibonacci(n):
if n <= 1: if n <= 1:
@ -136,6 +139,85 @@ def bubble_sort(size):
return arr[0] return arr[0]
def quick_sort_partition(arr, low, high):
pivot = arr[high]
i = low - 1
for j in range(low, high):
if arr[j] < pivot:
i += 1
arr[i], arr[j] = arr[j], arr[i]
arr[i + 1], arr[high] = arr[high], arr[i + 1]
return i + 1
def quick_sort_helper(arr, low, high):
if low < high:
pi = quick_sort_partition(arr, low, high)
quick_sort_helper(arr, low, pi - 1)
quick_sort_helper(arr, pi + 1, high)
def quick_sort(size):
arr = [0] * size
for i in range(size):
arr[i] = size - i
quick_sort_helper(arr, 0, size - 1)
return arr[0]
def binary_search(arr, target, low, high):
if low > high:
return -1
mid = low + (high - low) // 2
if arr[mid] == target:
return mid
if arr[mid] > target:
return binary_search(arr, target, low, mid - 1)
return binary_search(arr, target, mid + 1, high)
def binary_search_bench(size):
arr = [i for i in range(size)]
total = 0
for i in range(0, size, 100):
result = binary_search(arr, i, 0, size - 1)
total += result
return total
def string_concat(iterations):
length = 0
for i in range(iterations):
s = "Hello"
s = s + "World"
s = s + str(i)
length += len(s)
return length
def bitwise_ops(iterations):
result = 0
for i in range(1, iterations):
result ^= (i << 1)
result |= (i >> 1)
result &= (i >> 2)
result ^= ~i
return result
def array_copy(size):
src = [i for i in range(size)]
dst = [0] * size
for i in range(size):
dst[i] = src[i]
return dst[size - 1]
def insertion_sort(size):
arr = [0] * size
for i in range(size):
arr[i] = size - i
for i in range(1, size):
key = arr[i]
j = i - 1
while j >= 0 and arr[j] > key:
arr[j + 1] = arr[j]
j -= 1
arr[j + 1] = key
return arr[0]
def main(): def main():
print("=== Python Comprehensive Benchmark Suite ===") print("=== Python Comprehensive Benchmark Suite ===")
@ -244,6 +326,48 @@ def main():
print(f"Bubble sort (5000 elements) = {bubble}") print(f"Bubble sort (5000 elements) = {bubble}")
print(f"Time: {bubble_time} ms") print(f"Time: {bubble_time} ms")
start = time.perf_counter_ns()
quick = quick_sort(1000)
end = time.perf_counter_ns()
quick_time = (end - start) // 1000000
print(f"Quick sort (1000 elements) = {quick}")
print(f"Time: {quick_time} ms")
start = time.perf_counter_ns()
insertion = insertion_sort(3000)
end = time.perf_counter_ns()
insertion_time = (end - start) // 1000000
print(f"Insertion sort (3000 elements) = {insertion}")
print(f"Time: {insertion_time} ms")
start = time.perf_counter_ns()
binsearch = binary_search_bench(100000)
end = time.perf_counter_ns()
binsearch_time = (end - start) // 1000000
print(f"Binary search (100K, 1K searches) = {binsearch}")
print(f"Time: {binsearch_time} ms")
start = time.perf_counter_ns()
strconcat = string_concat(50000)
end = time.perf_counter_ns()
strconcat_time = (end - start) // 1000000
print(f"String concatenation (50K) = {strconcat}")
print(f"Time: {strconcat_time} ms")
start = time.perf_counter_ns()
bitwise = bitwise_ops(10000000)
end = time.perf_counter_ns()
bitwise_time = (end - start) // 1000000
print(f"Bitwise operations (10M) = {bitwise}")
print(f"Time: {bitwise_time} ms")
start = time.perf_counter_ns()
arrcopy = array_copy(1000000)
end = time.perf_counter_ns()
arrcopy_time = (end - start) // 1000000
print(f"Array copy (1M elements) = {arrcopy}")
print(f"Time: {arrcopy_time} ms")
print("=== Benchmark Complete ===") print("=== Benchmark Complete ===")
if __name__ == "__main__": if __name__ == "__main__":