will learning AWS help?!

The answer to this question is, unfortunately, the same as most of the X vs Y questions: it depends.

It depends on what you are trying to achieve or it depends on what kind of tech stack the company you want to work for is using. There are loads of companies using AWS, I mean half the internet runs on AWS, so long run it may be a good thing to pick up on.

The company I work for uses AWS as well and I have almost zero knowledge on the subject, but this doesn't affect my day-to-day job at all. It's outside the scope of my role for the company.

Learning whole AWS whole stack might not be useful but try to learn tools that your organization uses or you think might proves beneficial to your organization.
Like in our case

1. We shifted our front end to s3 which minimizes costs
2. Uses ALB(Application Load Balancer) for heavy traffic and handling socket connection.
3. Build Pipeline and trigger email on success using Lamda function.
4. Cloudfront for assets optimization
5. Security features like domain/referer locking.

I find difficult to learn something as side project, as I spent most of the time working for organization so aligning my learning or skillset to organization work proves handy for me.
Hope it helps

Yes! But you can also go your whole career without ever learning it. Don't think of it as something that you will necessarily need.