[RLlib] fix: Support custom eval functions that return zero eval_results, env_steps, or agent_steps {}, 0, 0

[petern48]

Description

Custom evaluation functions are expected to return the following return values: eval_results, env_steps, agent_steps. Sometimes, users may want to return an output of {}, 0, 0 to skip the iteration (see the issue for an example). However, the code would raise a ValueError because it checks for falsy values. This PR updates the validation to check for the exact types instead.

Additionally, I found that the code would throw a KeyError when the custom evaluation didn't write to the metrics store. To handle this, I added a default={} to the peek call. The same peek call a few lines down already does the same.

Related issues

Fixed #61513

Additional information

The following validation logic was originally added in this PR: #45652. It looks like the intention of the PR was to validate the inputs, not to prevent 0 or {} values.

ray/rllib/algorithms/algorithm.py

Lines 1633 to 1637 in 556e206

	if not env_steps or not agent_steps:
	raise ValueError(
	"Custom eval function must return "
	"`Tuple[ResultDict, int, int]` with `int, int` being "
	f"`env_steps` and `agent_steps`! Got {env_steps}, {agent_steps}."

[gemini-code-assist]

Code Review

This pull request addresses an issue where custom evaluation functions could not return falsy values like an empty dictionary {} or 0. The changes correctly adjust the validation logic to check for types instead of truthiness, allowing these valid return values. A new test case has been added to verify this fix. The implementation looks correct and effectively solves the described problem.

_{Note: Security Review did not run due to the size of the PR.}

[petern48]

cc @simonsays1980 as original author