I am writing to share an experience I recently had while working on an automation system for a client.

The system is built using an Express server that serves a svelte-kit and flutter app from Railway, and security is always a top priority for me. That's why I implemented measures such as cors, helmet, and express-rate-limit to keep the system secure.

However, while going through the logs for the system, I noticed that the express-rate-limit had been triggered. This was surprising because the site is still in development and I am the only user. Upon further investigation, I discovered that someone had been trying to gain unauthorized access to the system. These attempts included trying to locate unprotected .env files on the routes, and some of the requests were made less than a second apart, which indicated that it was a malicious attack rather than an accident.

In response to this event, I changed all passwords and identified and fixed any bugs in the system that were discovered through the logs. I also implemented a persistent blacklist for rogue IPs to prevent future attacks of this nature.

This experience has reminded me of the importance of continuously monitoring and securing my systems, and the need to stay vigilant in protecting against potential threats. I hope this post helps others to stay aware of the potential risks and take the necessary precautions to keep their own projects secure.

How do you have your OS secured? Do you have it seating behind a firewall? Do you have it exposed to the internet?

airsaylongcome:
How do you have your OS secured? Do you have it seating behind a firewall? Do you have it exposed to the internet?

It is a serverless deployment on Railway.

One of my assumptions is that the attack happened via Railway (possibly an hacker has access to their systems and its not yet discovered).

I would be contacting their support via email soon.

Railway deploys their default client servers as Bleep dot up dot railway dot app where Bleep is the client’s identifier.

For production, it is expected to use a custom domain which replaces the default with your domain.

Since this project is still in development, i have not applied the custom domain.

Stop using malicious scripts nulled themes/files they won't listen.


How else would you explain someone trying to gain access to your totally irrelevant server with zero financial/non-financial benefits whatsoever if not that the Hack was already in a malicious file u added to the server.

Happens a lot with nulled scripts

truthCoder:


It is a serverless deployment on Railway.

One of my assumptions is that the attack happened via Railway (possibly an hacker has access to their systems and its not yet discovered).

I would be contacting their support via email soon.

this did not happen through railway, this is just a random guess automation which the attacker is trying to search for vulnerabilities.

You can see he tried several URLs including wordpress paths. He doesn't even know the kind of stack you used for the portal. So it's a guess work on automation which is normal for novice.

The attacker is not trying to gain access to railway, rather the person is trying to guess work vulnerabilities on your portal by automating the process for path traverser and other yadayada

Sleekcode:
Stop using malicious scripts nulled themes/files they won't listen.


How else would you explain someone trying to gain access to your totally irrelevant server with zero financial/non-financial benefits whatsoever if not that the Hack was already in a malicious file u added to the server.

Happens a lot with nulled scripts

Dont you think it would be wiser if you read the post before commenting from your already-formed biased mind?

Since you are the god of code, abeg tell me which theme or scripts i copied on a simple Nodejs Express server? Do you even know how serverless deploys work?

To you, the system has no value but not to the company that paid in USD for a fully integrated workflow automation system. Do you even know what workflow automation is?

I don’t copy codes and would not have been able to spot this if i had unnecessary or unknown scripts running. This is not a landing page for iya basira’s food canteen or an NFT scamming site. It is a custom built site and you might not believe it but some people do write their own codes.

Even if you know more than everyone here, dont you think it is better you share your unlimited knowledge than trying to ridicule?

Edit: Someone smarter has given a response above. You can do well to read it. Thanks

bedfordng:
this did not happen through railway, this is just a random guess automation which the attacker is trying to search for vulnerabilities.

You can see he tried several URLs including wordpress paths. He doesn't even know the kind of stack you used for the portal. So it's a guess work on automation which is normal for novice.

The attacker is not trying to gain access to railway, rather the person is trying to guess work vulnerabilities on your portal by automating the process for path traverser and other yadayada

Very true.

My main concern would be how he got the server link Could it be through crawling?

What are the possible mitigations against such since the APIs are not customer facing?

I dont think i can use an IP whitelist as railway does refresh intermittently.

Inputs welcome

truthCoder:


Dont you think it would be wiser if you read the post before commenting from your already-formed biased mind?

Since you are the god of code, abeg tell me which theme or scripts i copied on a simple Nodejs Express server? Do you even know how serverless deploys work?

To you, the system has no value but not to the company that paid in USD for a fully integrated workflow automation system. Do you even know what workflow automation is?

I don’t copy codes and would not have been able to spot this if i had unnecessary or unknown scripts running. This is not a landing page for iya basira’s food canteen or an NFT scamming site. It is a custom built site and you might not believe it but some people do write their own codes.

Even if you know more than everyone here, dont you think it is better you share your unlimited knowledge than trying to ridicule?

Edit: Someone smarter has given a response above. You can do well to read it. Thanks

The earlier you stop copying codes, the better for you.

Still don't see any reason why someone would want to hack your valueless app.

truthCoder:


Very true.

My main concern would be how he got the server link Could it be through crawling?

What are the possible mitigations against such since the APIs are not customer facing?

I dont think i can use an IP whitelist as railway does refresh intermittently.

Inputs welcome

well, there is really nothing you can do other than to make sure your URL paths doesn't lead to leaks on your site. Most of them uses random IP. Blocking it can still reduce the rate at which they tries though.

Also make sure your codes logic doesn't break or do other things when mix matched paths are inputed.


Vulnerabilities can come from any angle. So don't trust users inputs even when you think it's only your client that would use the app. Hackers doesn't really care who is using the app. This is why they use automation tools to test most of the payloads they have succeeded in using to game other system before.

Sleekcode:



The earlier you stop copying codes, the better for you.

Still don't see any reason why someone would want to hack your valueless app.

You opinion has been noted.

You can now move along

Thanks

truthCoder:


You opinion has been noted.

You can now move along

Thanks

Move along to where?

I pin here gidigba.

You must stop using nulled scripts.


We know it's very addictive but u just have to stop

bedfordng:
well, there is really nothing you can do other than to make sure your URL paths doesn't lead to leaks on your site. Most of them uses random IP. Blocking it can still reduce the rate at which they tries though.

Also make sure your codes logic doesn't break or do other things when mix matched paths are inputed.


Vulnerabilities can come from any angle. So don't trust users inputs even when you think it's only your client that would use the app. Hackers doesn't really care who is using the app. This is why they use automation tools to test most of the payloads they have succeeded in using to game other system before.

All unrecognised paths resolve with a res.json() response, as long as they are within the rate limit. All requests above the rate limit simply fail and block the ip automatically.

I also have my error logics in place for all valid paths to reduce crashes.

I have zod and validator in place for input validations.

I also have my headers with custom tokens from my web and mobile apps for all requests which is verified via a middleware. This is in addition to the jwts for protected routes.

I am considering logging all rouge ips and storing them in a central blacklist for all my projects but the load time cost is what i am concerned about. Maybe if i create the list centrally, then have my projects have cron jobs that fetch from this list and update locally to a redis cache? Then cross referencing via the locally updated list should not cost more in time and compute.

@truthCoder
Just change the random url that Railway assigns you. And stop worrying too much.
When it's time to deploy to production and handover; deploy a sinkhole alongside and you can sleep easy.

Sleekcode:



The earlier you stop copying codes, the better for you.

Still don't see any reason why someone would want to hack your valueless app.

Can you see how this aged?

I brought down your own site.

Turned the crap into a white only page.

Lol

truthCoder:


Can you see how this aged?

I brought down your own site.

Turned the crap into a white only page.

Lol

....

Sleekcode:



Just woke up this morning to see your garbage screenshot.


Which site exactly did u bring down?


As at the time of me waking up (11:50AM) my website is still live lol.

Keep on dreaming and living in fantasy land... wannabe hacker lol


Mr Quack!

A liar like you?

Dont worry, i am creating a thread where i share your codes that i cloned before you made the repo private.

Wannabe programmer even exposed his .env files on a public repo.

Sha hold on

truthCoder2:


A liar like you?

Dont worry, i am creating a thread where i share your codes that i cloned before you made the repo private.

Wannabe programmer even exposed his .env files on a public repo.

Sha hold on

...

truthCoder2:


A liar like you?

Dont worry, i am creating a thread where i share your codes that i cloned before you made the repo private.

Wannabe programmer even exposed his .env files on a public repo.

Sha hold on

You're emphasizing too much on this .env file lol


Is that your big reveal?


Is that your definition of a hack?


Do u know the meaning of a hack?

...

Update:

I have created a whitelist and using cors(), i bounce off every request not coming from my apps.

I have created a blacklist where IPs that are running scripts on the endpoint are added. This array is global and prevents access to all resources permanently.

I have also updated the endpoint to a custom endpoint instead of the generic one issued by railway.

Finally, i have secured every single route with an additional token layer.

Tell you an experience we had at work yesterday. (Yes some of us started the grind yesterday). So because I'm in telephony, our systems are frequent targets, especially by bots from China and.....Palestine. The aim of the bots are to initiate international calls to premium numbers abroad. Think "seks" hotlines and OF lines. Or to Palestinian mobiles (don't know how the Palestinians profit from this but they are huge). So my colleague in Buenos Aires (lawyer turned techie) opened the well-known RDP port on his home router so he can access his work PC while way from home. Omo! Dem run the guy street. They used up his voip.ms account leaving him with a debt of $2k. I accomplish the same scenario (RDP access to my work PC) using a software service called Tailscale. Now, I have to train the rest of my colleagues how to use Tailscale.

So I fully understand how frustrating it would have been for you dealing with the hack attempts

airsaylongcome:
Tell you an experience we had at work yesterday. (Yes some of us started the grind yesterday). So because I'm in telephony, our systems are frequent targets, especially by bots from China and.....Palestine. The aim of the bots are to initiate international calls to premium numbers abroad. Think "seks" hotlines and OF lines. Or to Palestinian mobiles (don't know how the Palestinians profit from this but they are huge). So my colleague in Buenos Aires (lawyer turned techie) opened the well-known RDP port on his home router so he can access his work PC while way from home. Omo! Dem run the guy street. They used up his voip.ms account leaving him with a debt of $2k. I accomplish the same scenario (RDP access to my work PC) using a software service called Tailscale. Now, I have to train the rest of my colleagues how to use Tailscale.

So I fully understand how frustrating it would have been for you dealing with the hack attempts

Wow.

There are a lot of chinese routers out there with known backdoors which is available for sale on the dark web.

One of the greatest threats to cybersecurity currently is not just TikTok but these routers.

They are sold for very cheap and are currently deployed in the millions worldwide.

The best approach is to treat everyone else apart from you as a potential intruder and gate-keep access to allow only those you explicitly allow and for the resources you have granted access. No extras.

truthCoder:


Wow.

There are a lot of chinese routers out there with known backdoors which is available for sale on the dark web.

One of the greatest threats to cybersecurity currently is not just TikTok but these routers.

They are sold for very cheap and are currently deployed in the millions worldwide.

The best approach is to treat everyone else apart from you as a potential intruder and gate-keep access to allow only those you explicitly allow and for the resources you have granted access. No extras.

I only use pfsense routers that I assembled myself
I no get power abeg

truthCoder:
Update:

I have created a blacklist where IPs that are running scripts on the endpoint are added. This array is global and prevents access to all resources permanently.

Quick question: how do you intend to handle this blacklist? because it will grow exponentially over time. It's easier to manage a whitelist because it will have fewer entries to traverse than a blacklist with say, 200k IP addresses, which means on every request, you will be potentially looking up 200k ip addresses?

if you already have a whitelist, then there is no need for a blacklist; or what is the need? i want to hear your thought process.

airsaylongcome:
Tell you an experience we had at work yesterday. (Yes some of us started the grind yesterday). So because I'm in telephony, our systems are frequent targets, especially by bots from China and.....Palestine. The aim of the bots are to initiate international calls to premium numbers abroad. Think "seks" hotlines and OF lines. Or to Palestinian mobiles (don't know how the Palestinians profit from this but they are huge). So my colleague in Buenos Aires (lawyer turned techie) opened the well-known RDP port on his home router so he can access his work PC while way from home. Omo! Dem run the guy street. They used up his voip.ms account leaving him with a debt of $2k. I accomplish the same scenario (RDP access to my work PC) using a software service called Tailscale. Now, I have to train the rest of my colleagues how to use Tailscale.

So I fully understand how frustrating it would have been for you dealing with the hack attempts

What a bad situation to be in.....
Security 101 states that don't allow every tom, d and harry inbound IP address into a secure network. If he had opened port 3389, then he should have locked it to an IP addresses or ip addresses he trusts. The only major drawback here is having to deal with dynamic IPs from ISPs and in such a scenario, just get a static VPN as a secure tunnel back to the private resource. I run piHole at home, and since port 53 has to be publicly accessible to resolve, it is blocked and only allows responds to request from from my VPN. So to use the piHole, you gotta connect to the vpn first. if my ISP was static, i wouldn't need the vpn. It is what it is.

QuoteJustOnce:


Quick question: how do you intend to handle this blacklist? because it will grow exponentially over time. It's easier to manage a whitelist because it will have fewer entries to traverse than a blacklist with say, 200k IP addresses, which means on every request, you will be potentially looking up 200k ip addresses?

if you already have a whitelist, then there is no need for a blacklist; or what is the need? i want to hear your thought process.

You are very right.

If the blacklist should grow to a huge size, then i don get serious timeout problem be that.

I created a blacklist so that i can harvest any potential rouge IP for later analysis and inspection. The blacklist would serve as a log of all rouge IPs, if any that exists.

This would allow me know if any attempt is being made against the server and by which IP.

However, i am not expecting the blacklist to grow. A length of 0 is hoped for. If i have 200,000 blacklisted IPs, then blacklist size is no longer my main problem.

Instant checks of the blacklist (i get an email any time an entry is made into the blacklist) would hopefully assist to catch this at the budding state.

I thought as much. Well in that case, what you want is a log drain or log sink; as it's technically called. AWS has something called CloudWatch. Not saying you should use it though, but mentioned it as a pointer to what you need.
There's also DataDog and New Relic. Just connect your app to any of those and let the traffic flow through them. You don't need to maintain a blacklist, as they will log and sort every request for you in very nice formatted charts and what nots; which could be exported; so you can analyse all you want.

QuoteJustOnce:
I thought as much. Well in that case, what you want is a log drain or log sink; as it's technically called. AWS has something called CloudWatch. Not saying you should use it though, but mentioned it as a pointer to what you need.
There's also DataDog and New Relic. Just connect your app to any of those and let the traffic flow through them. You don't need to maintain a blacklist, as they will log and sort every request for your in very nice formatted charts and what nots; which could be exported; so you can analyse all you want.

Ok. I would look into your suggestions now.

Thanks so much.

truthCoder:


Ok. I would look into your suggestions now.

Thanks so much.

Anytime, boss!

Na latin i dey read so. Omo

Sleekcode:
Stop using malicious scripts nulled themes/files they won't listen.


How else would you explain someone trying to gain access to your totally irrelevant server with zero financial/non-financial benefits whatsoever if not that the Hack was already in a malicious file u added to the server.

Happens a lot with nulled scripts

That might not be the case here. And moreover, there are bots all over the internet and all they do is randomly hit on what they could find.