dah85.com

*not* just another blog ;)

Quick Tip

Quick tips

I've been using an IPTV service that works great during the day, but at night it skips and stutters and buffers like crazy. It turns out the reason for this is to do with routing.

In Australia - specifically on the NBN - peak hour is usually between 3PM and 10PM, roughly when school kids start getting home and people get home from work and start their Netflix binge watching etc.

Most (if not all?) IPTV providers tend to have their servers in Europe. It seems like Germany is a popular choice for this. I have a 100mbps (megabit per second) connection at home and during off-peak I can get around 300 kilobytes per second which is 0.3mbps (0.3 megabit per second) and during peak time this plummets down to around 50kb/sec (kilobytes per second) or 0.05mbps (megabits per second) and this makes IPTV unwatchable.

After testing different speed tests from different locations, I've realised that Australia has a pretty consistent connection to the USA, in particular LAX and Miami during off-peak and even during peak times. It got me thinking about using that to my advantage to help with my IPTV situation.

How I fixed it

I decided that the best approach here would be to set up a VPS in an ideal location, run a proxy server on it and point VLC to that proxy server.

The first thing I did was set up a cheap VPS in Los Angeles. I chose to use this one for $8/year.

For better results and to be able to select different locations or change IP etc, I would recommend setting up a cheap VPS resource pool. This one is $19/year and lets you create up to 2 VPS in 4 locations in the USA (LA, Miami, New York and Chicago.

Once I had that set up, the next thing I did was set up a tiny proxy server, aptly named Tinyproxy.

Setting up Tinyproxy

I am setting this up on a Ubuntu 16.04 VPS, if you are using another distro then replace apt with yum/etc:

sudo apt update
sudo apt install tinyproxy

Tinyproxy is now installed, but it needs to be configured.

sudo nano /etc/tinyproxy.conf

We need to change a couple of things. First, the port. It defaults to 8080 which is very predictable and likely to result in your server being used by someone else. Change it to a random high port number that you will remember:

# Port: Specify the port which tinyproxy will listen on.  Please note
# that should you choose to run on a port lower than 1024 you will need
# to start tinyproxy using root.
#
Port 8080

Secondly, by default, Tinyproxy is set to only allow access from the computer it's running on - not ideal in this case. Find the following lines and change it to look like this:

# Allow: Customization of authorization controls. If there are any
# access control keywords then the default action is to DENY. Otherwise,
# the default action is ALLOW.
#
# The order of the controls are important. All incoming connections are
# tested against the controls based on order.
#
#Allow 127.0.0.1
#Allow 192.168.0.0/16
#Allow 172.16.0.0/12
#Allow 10.0.0.0/8

Note I have placed a # in front of Allow 127.0.0.1. By doing this I have made it accept all connections. It would be better to set up a rule to allow only a certain range, but for simplicity's sake I am leaving it like this.

Now that we've configured Tinyproxy, we need to restart the VPS.

sudo reboot

When the VPS comes back online it's all ready to go and Tinyproxy is waiting for us to use it.

Configure VLC Player

Finally, we need to configure VLC player to use this proxy when playing the videos.

Go to Tools then Preferences and select the Input/Codecs tab. Right down the bottom you will see HTTP Proxy URL

Go ahead and put the IP of your VPS in there along with the port. For example 123.123.123.123:8080 making sure to replace the IP with your VPS IP and the port with the port you've chosen in the config file. Once that's in there, click Save.

Testing it

The moment of truth is upon us. Go ahead and try watching the IPTV stream that you usually have issues with, it's best to test it during off-peak time and as well as during peak time to be sure it's set up correctly.

And there we have it, we've got flawless* IPTV :)

*of course, other factors can come into play, but this will resolve issues related to routing.

Showing disk usage in Linux

- Posted in Quick Tip by with comments

If you are wondering what is taking up all the space on your Linux server or computer, then there is a simple, handy tool called ncdu which shows you exactly how much space is being used by each folder.

sudo apt install ncdu

Then if you want to view the entire computer:

cd /
ncdu

Or if you want to see which is the biggest folder in your Transmission download folder..

cd /var/lib/transmission-daemon/downloads
ncdu

To exit, press q.

Using a VPS as a proxy with SSH

- Posted in Quick Tip by with comments

Just a quick tip, if you want to use a VPS as a proxy server through SSH then use this command (if your computer runs Linux)

ssh -D 12345 [email protected]

Replace 12345 with the port you want to use.

Then in Firefox, go to Preferences and add localhost as a Socks 5 proxy and the port 12345

I always test it first by going to Google and searching "What's my IP" to make sure it shows as the VPS's IP.

And you're done.

On one my test servers running VestaCP on an LXC container, I ran into an interesting issue where (due to the nature of LXC this is expected behaviour) the VestaCP LXC container shows the load of the host node, and as it's under load, VestaCP refuses to perform backups.

If you get an email like this "LoadAverage 5 is above threshold" and you would prefer to change that to something a bit higher (in my case, I'm changing it to 20 because I want it to backup even if the server is on fire) then here's how to do it.

We need to edit the BACKUP_LA_LIMIT=5 variable in /usr/local/vesta/func/main.sh

sudo nano /usr/local/vesta/func/main.sh

In my case I've changed it to read BACKUP_LA_LIMIT=20 so that it will always backup.

I searched high and low on the internet and couldn't find anything that suggested this, so I ended up doing a grep -r "BACKUP_LA_LIMIT" in the /usr/local/vesta directory to find which file is responsible.

I hope this can help someone else :)

Proxmox allows a lot of powerful options when it comes to shared file systems, but nothing could be simpler than using sshfs and adding the directory to Proxmox as a storage option for backups, VMs and ISOs.

In my case, I have 2 Proxmox servers and a 3rd server running Ubuntu which I would like to store backups on.

I my 2 Proxmox servers which are an i5 and an i7 from SerweryDedyKowane.pl which are both hosted on the same network in Poland, and the backup server is an i5 V-Dedi from Wishosting which is hosted in Canada.

Here's what I did:

First, I set up sshfs to point to the storage on the backup server.

In this example, I have set up sshfs on both Proxmox servers. So, on each Proxmox server there is a folder called /mnt/remotebackup which is where I'd like to store my backups.

In the Proxmox UI, we need to click on Datacenter then click on Storage. From there, we need to Add a Directory.

  • I gave mine the ID of remotebackup
  • For Directory, I put in /mnt/remotebackup which if you're following this example, feel free to change to what you're using.
  • For Content, I have selected VZ Dump Backup File which allows me to store backups there. I have not tried using it to store anything else, but I plan to test that out at some stage.
  • Nodes are set to all, Enabled set with a tick, same with Shared.
  • I have set Max Backups to 5, but you can change that as you wish. This is the number of copies of the backup it will keep before it starts deleting the old ones to make room for new ones.

Now that we have that set up, on BOTH Proxmox servers (In my case, it actually copied across by itself so you may only need to do it once, but if not, do it on all the other Proxmox servers) - we can start using the storage.

To verify that backups will work, I will select a VM, and then go to the Backup tab and click on Backup Now - on this screen make sure that Storage is showing the new storage we made, in my case it's called remotebackups

Go ahead an make a backup, it'll take a little while depending on the size of the VM but it will be stored on the remote server.

Neat eh?

Setting up SSHFS

- Posted in Quick Tip by with comments

Today I will be setting up SSHFS, which will allow me to mount a remote folder as a local folder by using SSH.

In this example, I will be creating a folder called remotestorage which will link to a remote folder on a remote server.

First thing we need to do is install SSHFS:

sudo apt update & apt install -y sshfs

Then we need to create the local mount point, just like this:

sudo mkdir /mnt/remotestorage

Now, we can manually mount the remote folder:

sudo sshfs -o allow_other [email protected]:/ /mnt/remotestorage

Log in as requested, and you should be able to see the contents of your remote folder.

If you want to unmount, we just need to do this:

sudo umount /mnt/remotestorage

It's possible to have it automatically mount when the system starts, but it's a bit of a security risk and I wouldn't recommend it.

Remove a node from Proxmox 5

- Posted in Quick Tip by with comments

I have a Proxmox cluster set up using a few dedicated servers on the same network. I have decided to downsize my cluster and remove 2 of the nodes, leaving only 2 behind.

Normally, to remove a node from Proxmox it's a simple matter of shutting down the node to be removed and then on one of the Proxmox cluster members, removing it like this:

pvecm nodes

That will list the nodes in the cluster

pvecm delnode <nodename>

Which will remove it from the cluster.

However, when I went to do that I was given an error:

cluster not ready - no quorum?

Ruh roh! A simple fix is to let Proxmox know that there are only 2 servers left in the cluster, therefore the number of "votes" the quorum needs to do things will be set.

pvecm expected 2

Then try pvecm delnode again:

pvecm delnode testnode3

The output you will see if succesful will be similar to this:

Killing node 3

To remove them from the web UI, we need to delete the folders like this:

rm -rf /etc/pve/nodes/nodename

And we're done :)

I have a solution for the annoying issue you may encounter when using a Raspberry Pi with a USB mouse (both wired and wireless) where the mouse movement seems either jerky, laggy, erratic, slow, laggy etc.

What you need to do is take the SD card out of the Pi and put it into your main computer, and then open the BOOT partition, then look for a file named:

cmdline.txt

You will see a line in that file, what we need to do is add the following to the END of that line. NOT on a new line. It must be at the end of the line:

usbhid.mousepoll=0

Save, and safely unmount or eject the SD card. Pop that back into the Pi and boot up, you should have nice mouse movement now :)

A quick tip so that I don't forget this again. By default, VestaCP will set directories to disallow listings, for example, example.com/files would show a 403 error instead of listing the contents.

Sometimes, though, it's desirable to have a list shown to the user, so all we need to do is create or modify the .htaccess file in the top level folder.

For example, in /home/user/web/example.com/public_html/files/

nano .htaccess

Add the following to ALLOW listing:

Options +Indexes   

Or to DISALLOW:

Options -Indexes   

Hopefully this also helps someone :)

Today I will be setting up SSL certificates for Proxmox 5 so that when you go to the web UI, it will be HTTPS and not using the self-signed cert that comes with Proxmox, which is rather insecure.

I will be doing this with Certbot.

First, we need to install Certbot:

apt install certbot -y 

Now, we need to set up the domain we're using for PVE and obtain a certificate:

certbot certonly

I will be using option 2, to spin up a temporary webserver so that certbot can verify that the domain points to the IP of the Proxmox server.

Now, we need to copy the cert files into the Proxmox directory like this:

cp /etc/letsencrypt/live/**yourdomain.com**/fullchain.pem /etc/pve/local/pveproxy-ssl.pem
cp /etc/letsencrypt/live/**yourdomain.com**/privkey.pem /etc/pve/local/pveproxy-ssl.key

And when that's done, we need to refresh Proxmox so it can be aware of the changes:

systemctl restart pveproxy

You should be able to see that it's now accessing through HTTPS and with a valid certificate - no more warnings :)

We need to make this permanent, so we'll create a cron job to keep it updated and renew the cert as needed:

crontab -e

Then paste the following on a new line:

30 6 1,15 * * root /usr/bin/certbot renew --quiet --post-hook /usr/local/bin/renew-pve-certs.sh

Control-X to exit, Y to save and press Enter to save the file with the original name.

And we're done :)

I'm going to add something extra here because it might apply to you too, but if you're also running VestaCP on your Proxmox server with port 80 and 443 forwarding to your VestaCP server, the certbot method shown will fail - what we need to do is set up the PVE domain in VestaCP first, which will work, then copy the files from VestaCP to Proxmox and then follow the steps. I'll clarify this if someone comments requesting more details.