It could very well be an A1111 thing if that's what you're using. As mentioned, I use ComfyUI and set it to use the SDXL workflow. Even the normal SD1.x workflow on ComfyUI comes up with really wild results sometimes, though it can also depend on what sampler method you're going with. Although some of the model checkpoints tell you to use Eular with Karras, try experimenting with the other sampler methods. For my example, I use DDIM with Karras at 20 steps, and left CFG at 5. I only use Illustrious model checkpoints now because they're trained specifically for anime, so perhaps it's the checkpoint model you're using and/or sampler method? Or perhaps it's because you're leaving in generated positive prompts like 'photorealism'.
From my experience, I only get the green olive skin goblin slaves if I leave the 'photorealism' prompt in when I set my own Override Positive Prompts. I don't bother with the pose and expression prompts because I feel like when you set Override Generated Prompts, it doesn't even register what you enter in those fields and only reads off what I entered into the Override Positive and Override Negative prompt fields. One way to really find out if it's doing what you think it should be doing is to copy the Positive and Negative prompts into A1111/ComfyUI/Forge with the same model checkpoint, sampler method and type to see if it matches what comes out in the game itself. With ComfyUI and the Save Extended extension at least, I made it so that every model checkpoint has its own subfolder in the output folder so that it's easier for me to test the same prompts with different model checkpoints to see what changes, and see which one looks better to me. On that note, I've tested Cat Tower, Hassaku XL Illustrious, Illustrious Pencil XL, NoobAIXL (Pred and vPred versions), Nova Anime XL, Obsession Illustrious XL, SweetMix Illustrious XL and Wai NSFW Illustrious. Nova Anime XL for me resulted in a pretty consistent anime style; all the outputs from it in FC so far look like as if they've been (AI) drawn by the same (AI) artist, even if I get it to start drawing serafuku or halterneck dress outfits instead of kimonos. The civitai page itself tells you to use Eular a, but Eular a is known for having extremely varying results if something isn't right. I'd rather normal Eular over Eular a, but otherwise for me DDIM works well enough in most cases and it's pretty fast as well.
