Well, therein lies the sort of question that has come up many times.
Shell scripting ranges from common basics to massive expanse as there are various flavours (of shell) and hundreds, if not thousands of tools, available for the command line.
AppleScript is a scripting language, just like Python, or Go, or numerous others.
Java is a full on programming language, but I suspect what you actually meant was JavaScript; which is different again, but you will see come up lots in these forums.
There is often overlap between tolls, platforms and languages. But at the same time, specific tasks are certainly easier in some than in others.
For starting out, I would suggest learning some shell scripting. It will open up a lot of opportunities and even a small amount can make a big difference. You can use it inside Keyboard Maestro, inside Automator and inside AppleScript, if and when the time comes.
As you need to do more things, learn more. But, avoid people’s usual problem of trying to do everything at once. Build up the solutions and build up your tools one step at a time.
When you find something that is difficult to do in shell script and you feel like it is becoming too difficult or you have come to an impasse, start casting your sights farther afield. What you have learned is cumulative and what you learn for one tool you will often find at least some similarities to in other tools. Things like variables, loops, and logical conditions are practically universal.
Hope that helps.