How to Disable or Enable open relay in sendmail
Sendmail 8.9 and above will disallows relaying by default. By default this will allow relaying from localhost only.
You can edit the sendmail access database ( /etc/mail/access) and enable or disable open relay.
You can use host name or ip address in the access database
firstname.lastname@example.org 550 Spammers not allowed
To Reject Mail Form Particular Hoste
Open Relay servers will be configured to relay all ip range
By adding the *.* entry in /etc/mail/access you can enable relay for all
Now a days most of the mail servers will not allow mails from open relay servers.
To Disable OPEN Relay:
By allowing relay from only your local ip range and localhost, you can disable the open relay.
If your server is configured with smtp auth or POP-before-SMTP, then you can remove all the relay entries (Even local ip range) in the access database.
It is always better to configure your server with smtp auth or POP-before-SMTP . This will be useful for roaming users also.
Also check /etc/mail/relay-domains and remove all the relay domains
In some cases relaying is allowed even though it isn't intended. This is because of the use of FEATURE(relay_entire_domain) which allows relaying for every system in class m. If class m is wrong, then the system may open up relaying for other hosts in that domain. You have to verify this
echo '$=m' | sendmail -bt -d0.4
Sometimes class m is set wrong to be just a TLD (e.g., com). In this case, fix your hostname to be host.sld.tld instead of sld.tld or change $m, or remove FEATURE(relay_entire_domain) and use other means instead
Open Relay Testing Websites:
smtp authentication vs. pop before smtp