Portable Testsuite to check a C / C++ Compiler and standard runtime libraries

Many compiler and the related standard libraries provide tests, but they are not easy usable as portable/standalone conformance check for other compiler or for other runtime library implementations.

Goals for stdtests

• [ ] A reusable license

• [ ] Only needs a Compiler and a posix shell to run the testsuite

• [ ] Portable: Works on many systems with many compiler and runtime libraries

• [ ] Check C Compiler and the C Runtime library

• [ ] Check the POSIX Runtime library

• [ ] Check C++ Compiler and the C++ Runtime library

• [ ] Create a nice Table from multiple Results

• Currently, only headers are checked for availability during configure

Expected usecase

• Extract the testsuite
• configure the testsuite to test a compiler / runtime library for a given standard
• make: run the testsuite and generate a result file
• make table: generate a table for multiple results

Expected usage

• std: newest
• Compiler: tcc
• runtime-library: system-default

mkdir results
cd results
CC=tcc ../configure --disable-c++
make

• Now there should be an document with the results of the current test

Portability

• Current tested environments |Target OS|tested Compiler| |:-:|---| |Linux| tcc / gcc / g++ / owcc / clang / zig cc / zig c++| |Windows| x86_64-win32-tcc / x86_64-w64-mingw32-gcc / x86_64-w64-mingw32-g++|

• Hints / Links for later use:

  • With many years delay, a command line switch arrived in MSVC 2015 Update 3 for selecting the c++ version

Checking C Compiler and the C Runtime library

|STD| C2y | C23 | C17 | C11 | C99 | C95 (C90+AMD1) | C89/C90 / ANSI-C| |:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|

Checking POSIX Runtime library

Currently, POSIX.1-2024 ist referenced. Older versions will follow.

Checking C++ Compiler and the C++ Runtime library

|STD| C++2c | C++23 | C++20 | C++17 | C++14 | C++11 | C++03 | C++98 | |:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|

Some examples of other tables with Features or Test-Results

• gcc Status of C99 features in GCC / C11 Status in GCC / C++ Standards support in GCC
• clang C Support in clang / C++ Support in Clang
• msvc C/C++ Support in MSVC
• dmc C++0x language Implementation
• owcc C99 Compliance / C++ Language / C++ Library / C++ Library Status
• cppreference.com What's new in C99 / C99 Implementation status / C23 Implementation status / C++ Implementation status

Some links to compiler / library tests

|Name|URL |:-:|:-:| |clang|test top dir| |gcc|test top dir| |mcpp|preprocessor| |musl|libc-tests| |glibc|libc|

---

C Reference Documentation

• ISOC SC22/WG14 C standards commitee
• ISO SC22/WG14 docs
• ISO SC22/WG14 document log
• C Project status and milestones
• C23: Early C2y draft from 02.02.2024 was N3220
• C17: Early C23 draft from 11.11.2018 was N2310

Available C Standards

<details><summary>Released C Standards (Click Arrow to expand)</summary>

|std|__STDC_VERSION__|Latest Draft|Release| |:-:|:-:|:-:|:-:| | C2y | N/A | N3550 | N/A | | C23 | 202311 | N3096 | ISO/IEC 9899:2024 | | C17 | 201710 | N2176 | ISO/IEC 9899:2018 | | C11 | 201112 | N1570 | ISO/IEC 9899:2011 | | C99 | 199901 | N1256 | ISO/IEC 9899:1999 | | C95 | 199409 | c89_na1: c94 | ISO/IEC 9899:1990/AMD1:1995 | | C90 | not defined | FIPS 160 | ISO/IEC 9899:1990 | | C89 | not defined | c89 draft | ANSI X3.159-1989 |

</details>

---

POSIX Reference Documentation

POSIX (Portable Operating System Interface) specifies many parts of an Operating System.

• Unix standards overview

Interesting for this project is the API part of the Base Specification.

Available POSIX Standards

<details><summary>Released POSIX Standards (Click Arrow to expand)</summary>

The Open Group Base Specification is also released as the Single UNIX Specification and also released as the ISO/IEC 9945-1 standard.

|_POSIX_VERSION|_XOPEN_VERSION|The Open Group Base Specification|View|Downloads| |:-:|:-:|:-:|:-:|:-:| |202405|800|POSIX.1-2024, Issue 8, 2024 Edition|View online|susv5| |200809|700|POSIX.1-2017, Issue 7, 2018 Edition|View online|susv4-2018| |200809|700|POSIX.1-2008, Issue 7, 2016 Edition|View online|susv4tc2| |200809|700|POSIX.1-2008, Issue 7, 2013 Edition|View online|susv4tc1| |200809|700|POSIX.1-2008, Issue 7, 2008 Edition|View online|susv4| |200112|600|POSIX.1-2001, Issue 6, 2004 Edition|View online|susv3| |199506||POSIX.1-1996| | | |199009||POSIX.1-1990| |FIPS 151-2 | |198808||POSIX.1-1988| |FIPS 151-1 |

• Single UNIX Specification (SUSv*)

  • UNIX V8, Version 5
  • UNIX V7, Version 4
  • UNIX 03, Version 3
  • UNIX 98, Version 2
  • UNIX 95, Version 1

• POSIX Specification

  • ISO/IEC/IEEE 9945:2024
  • ISO/IEC/IEEE 9945:2009
  • ISO/IEC/IEEE 9945-1:2003
  • ISO/IEC/IEEE 9945-1:2002
  • ISO/IEC/IEEE 9945-1:1996
  • ISO/IEC/IEEE 9945-1:1990

</details>

---

C++ Reference Documentation

• ISOCPP SC22/WG21 C++ standards commitee
• C++ standard core language issues list
• C++ standard library issues list
• C++26 First draft was N4958
• C++23 is available at the iso.org Website since 19.Oct.2024

Available C++ Standards

<details><summary>Released C++ Standards (Click Arrow to expand)</summary>

|std|__cplusplus|Latest Draft|Release| |:-:|:-:|:-:|:-:| |C++26| N/A |N5014|N/A| |C++23|202302|N4950|ISO/IEC 14882:2023| |C++20|202002|N4849|ISO/IEC 14882:2020| |C++17|201703|N4659|[ISO/IEC 1