Engine and animations are not really related. The engine will tell you more about the gameplay experience.
Renpy for example is an engine for making Visual Novels.
Good Girl Gone Bad is a a well made Renpy game and representative of what kind of experience you'll have playing that kind of game.
Naked Ambition is a great example of how uniquely different a Renpy game can be though. As it is a visual novel engine, it is also likely to attract certain visual styles - still images, be they 2D or 3D work well here.
RPGMaker is an engine to create top down old school rpgs, reminiscent of the old Final Fantasy, Chrono Trigger games etc. Games made in this engine usually involve a lot of running around, which does not always work well in an adult game that doesn't really involve exploration and random combat encounters like a Final Fantasy game. As it is a pixel graphics engine, porn is usually done on another layer, using the game's messages/talk to have the sex pop up over the map. As RPGMaker allows for a world for the player to walk around, it is often chosen for more ambitious projects.
A New Dawn is a good example of an ambitious RPGMaker game, that needs the freedom the engine provides.
Claire's Quest tells a very similar story to A New Dawn, but the art direction is very different.
Karryn's Prison is a good example of how much can be gotten out of the engine.
One problem with RPGMaker games is that they tend to have enormous install size.
HTML (Twine, Sugarcube, or whatever) is good for creating very lightweight games with an interface akin to a website. It is probably the most likely candidate for games that use real porn images and videos. Games can have choices and multiple paths.
Suddenly A Girl is a good example of an HTML game.
Unity is a professional, very versatile engine. In the non-adult gaming world it is used for rpgs, tactitcs and all kind of games. Battletech comes to mind. For adult games here you have Tomie Wanna Get Married while at the same time there are
Orc Massage,
Pizza Hot! or
Escape Dungeon and
Cyberpink: Tactics .
To make matters more confusing, there are all the games that are created in RPGMaker, then get remade in Renpy
Once you play games more aware of the engines, you can use that information to decide if they will provide a fun experience. But you will have to check the screenshots in the first post and tags at the top of the threads if you would enjoy the visuals. I for example don't usually play RPGMaker games - too much time to download and too much running around. If I like the art I may wait for a Renpy remake such as Anne Exciting Affection.
If you have found a specific art style or animation style you really like, there aren't really any keywords to search for. You can use the forum search function with just the game's name - it is possible someone else looked for comparisons before you. And you can ask for help finding stuff if you can't find it yourself in the
Recommendation section.
That all said (and I hope it was at least somewhat helpful), Tomie is rather unique in every way - gameplay and visuals. So I do not know if you will find something truly similar.