Lexical Structure, Tokens & Literals

Before Java code has any meaning, the compiler breaks it into tokens — identifiers, keywords, literals, operators, and separators — the raw vocabulary every program is built from.

Lexical analysis is the first stage of compilation: the compiler scans raw source text and groups characters into tokens — the smallest meaningful units of the language. Java's tokens fall into five categories: identifiers, keywords, literals, separators, and operators, with whitespace and comments discarded (except as token boundaries). This topic covers that vocabulary in detail, especially literals, which represent fixed values written directly in source code.

Tokens Overview

Every Java program, before it is parsed into a syntax tree, is first broken into a stream of tokens. The five kinds of tokens are: identifiers (names like total or Account), keywords (reserved words like class or if), literals (fixed values like 42 or "hi"), separators (punctuation like { } ( ) ; ,), and operators (symbols like + - == &&).

Tokens here: keyword(int), identifier(total), operator(=), literal(42), operator(+), identifier(count), separator(;).

int total = 42 + count;

Whitespace and Comments as Token Separators

Whitespace (spaces, tabs, newlines) and comments are not tokens themselves — they are discarded by the lexer, but they matter because they separate tokens that would otherwise merge. int x differs from intx only because whitespace separates the keyword int from the identifier x.

Identifiers

Identifiers name variables, methods, classes, and other program elements. They are covered fully in the Keywords & Identifiers topic; briefly, they start with a letter, underscore, or dollar sign, followed by any combination of letters, digits, underscores, or dollar signs, and cannot match a reserved keyword.

Keywords

Keywords are reserved words with fixed meaning to the compiler (e.g. class, if, while, return, int) — they cannot be used as identifiers. The complete list and the distinction between reserved and contextual keywords is covered in the Keywords & Identifiers topic.

Separators

Separators are punctuation characters that delimit and structure code: ( ) { } [ ] ; , . ::

( ) — method parameters, expression grouping • { } — blocks • [ ] — array declarations/indexing • ; — statement terminator • , — separates list items (parameters, variable declarations) • . — member access • :: — method reference (Java 8+).

Operators (Preview)

Operators act on operands to produce values or effects — arithmetic (+, -, *, /, %), assignment (=, +=), relational (==, <, >=), logical (&&, ||, !), bitwise (&, |, ^, ~), and more. Operators are covered in full in their own dedicated topic; here they're mentioned only as one of the five lexical token categories.

Integer Literals

Integer literals can be written in decimal (base 10), hexadecimal (prefixed 0x or 0X), octal (leading 0), or binary (prefixed 0b or 0B, since Java 7). By default an integer literal has type int; appending L or l gives it type long.

Use lowercase l with care in source code — a lowercase L (l) looks like the digit 1 in many fonts; uppercase L is the conventional, safer choice.

int decimal = 42;
int hex = 0x2A;        // 42 in hexadecimal
int octal = 052;        // 42 in octal (leading zero)
int binary = 0b101010; // 42 in binary
long big = 3_000_000_000L; // needs the L suffix, exceeds int range

Floating-Point Literals

Floating-point literals default to type double. Appending f or F gives type float; d or D explicitly marks double (optional, since it's already the default). Scientific notation is supported with e or E.

double d1 = 3.14;
double d2 = 3.14d;      // explicit double suffix (redundant but legal)
float f1 = 3.14f;        // float suffix required, otherwise this is a double
double sci = 6.022e23;   // scientific notation
double sci2 = 1.5E-10;

Character Literals

A character literal is a single character enclosed in single quotes, representing one char value (a 16-bit UTF-16 code unit). Escape sequences represent characters that are hard to type directly.

char letter = 'A';
char newline = '\n';
char tab = '\t';
char backslash = '\\';
char singleQuote = '\'';
char unicodeChar = '\u0041'; // 'A' via Unicode escape

String Literals

A string literal is a sequence of characters enclosed in double quotes, representing a String object. String literals support the same escape sequences as character literals.

Java also has text block literals (triple-quoted, """), introduced as a standard feature in Java 15, for multi-line strings — covered in their own Modern Java topic.

String greeting = "Hello, World!";
String path = "C:\\Users\\name"; // \\ escapes a literal backslash
String quoted = "She said \"hi\"";

Boolean Literals

There are exactly two boolean literals: true and false. They are reserved words, not identifiers, and are the only valid values a boolean variable can hold directly as a literal.

boolean isReady = true;
boolean hasErrors = false;

The null Literal

null is a special literal denoting 'no reference' — the absence of an object. It can be assigned to any reference type variable, but never to a primitive type variable.

String name = null;   // OK, String is a reference type
int count = null;      // compile error: int is primitive, cannot hold null

Numeric Literal Separators (Underscores)

Since Java 7, underscores can be inserted between digits in numeric literals purely for readability — the compiler ignores them entirely. They cannot appear at the start/end of a number, adjacent to a decimal point, or adjacent to a type suffix.

int million = 1_000_000;
long creditCardNumber = 1234_5678_9012_3456L;
double pi = 3.14_159;

// invalid placements:
// int bad1 = _1000;      // cannot start with underscore
// int bad2 = 1000_;      // cannot end with underscore
// double bad3 = 3._14;    // cannot be adjacent to decimal point

Literal Suffixes Summary

L / l  -> long           (42L)
F / f  -> float          (3.14f)
D / d  -> double         (3.14d, optional/default)
0x/0X  -> hex prefix     (0x2A)
0b/0B  -> binary prefix  (0b101010)
leading 0 -> octal prefix (052)

Key Takeaways

Java source is lexed into five token categories: identifiers, keywords, literals, separators, operators; whitespace/comments are discarded but still separate tokens.

Integer literals default to int; use L/l for long. Floating-point literals default to double; use f/F for float.

Character literals use single quotes and hold one UTF-16 char code unit; string literals use double quotes.

true, false, and null are reserved literals, not identifiers.

Underscores may separate digits in numeric literals for readability, but cannot appear at the edges of the number or next to a decimal point/suffix.

float f = 3.14;

float f = 3.14f;

A floating-point literal without a suffix defaults to double; assigning it directly to a float variable is a compile error (narrowing without a cast) unless the f suffix is used.

int creditCard = 1234_5678_9012_3456;

long creditCard = 1234_5678_9012_3456L;

The literal's value exceeds int's range; it needs the L suffix to be treated as a long literal, and the variable itself must be declared long to hold it.

int leadingZero = 052; // programmer expects 52 int value = 52; // avoid a leading zero unless octal is intended

A leading zero makes an integer literal octal, not decimal — 052 is actually 42 in decimal, a frequent source of silent bugs.

What are the five categories of Java tokens?

Identifiers, keywords, literals, separators, and operators.

What type does an integer literal have by default, and how do you make it a long literal?

int by default; append L or l (L is preferred for readability) to make it a long literal.

Why does float f = 3.14; fail to compile, and what is the fix?

3.14 without a suffix is a double literal; assigning a double to a float is a narrowing conversion requiring either the f suffix (3.14f) or an explicit cast.

What value does the literal 052 represent, and why?

42 in decimal — a leading zero marks an integer literal as octal, so 052 is interpreted as octal 52, which equals decimal 42.

Where can underscores NOT be placed in a numeric literal?

At the very start or end of the number, immediately adjacent to a decimal point, or immediately adjacent to a type suffix.

Can null be assigned to a primitive variable like int? Why or why not?

No — null represents the absence of a reference and can only be assigned to reference-type variables; primitives always hold an actual value and have no 'no value' state.

What is the difference between the character literal 'A' and the string literal "A"?

'A' is a char literal representing a single UTF-16 code unit; "A" is a String literal representing a String object containing one character — they have different types and are not interchangeable.

What does this print, and why?

Prints '16 8 2'. 0x10 is hexadecimal for 16; 010 is octal (leading zero) for 8; 0b10 is binary for 2.

Write declarations for a long holding ten billion, a float holding 2.5, and a char holding a tab character, using correct literal suffixes/escapes.

long tenBillion = 10_000_000_000L; float halfMore = 2.5f; char tabChar = '\t';

10_000_000_000L needs the L suffix since it exceeds int's range; 2.5f needs the f suffix to avoid a double-to-float narrowing error; '\t' is the tab escape sequence.

What gets printed by this statement, and how many lines does the output occupy?

It prints across two lines: 'Line1' then 'Line2', because \n inside the string literal is an escaped newline character.

int a = 0x10;
int b = 010;
int c = 0b10;
System.out.println(a + " " + b + " " + c);

String s = "Line1\nLine2";
System.out.println(s);