\nFor work, I was asked to help port a project to Linux. This involved
\nworking on a computer behind a firewall. I am sure that every one is
\nfamiliar with ssh. For those of you who are not, ssh is a program that
\nencrypts a telnet session. And telnet is a program that allows someone to
\nlog into a computer remotely. So, to access the computer, all I would
\nneed to do is to ssh into it. Unfortunately, it was not that easy.\n<\/p>\n
\nMore follows:\n<\/p>\n
<\/p>\n
\nThe first problem is the firewall which exists to provide some measure of
\nsecurity. The computer that I wanted to access (bob), was behind a
\nfirewall and as a result was not accessable. To bypass the firewall, I
\nwas provided with an account on that machine. Once I logged into the
\nfirewall (bill), I could then turn around and access bob.\n<\/p>\n
\n
\nssh hamzy@bill
\n<\/code>\n<\/p><\/blockquote>\n\nand then from within the ssh session\n<\/p>\n
\n
\nssh hamzy@bob
\n<\/code>\n<\/p><\/blockquote>\n\nWhat if I wanted to run the second ssh on my own machine (localhost)?
\nFortunately, ssh provides a solution called port forwarding. Port
\nforwarding essentially is a small program the reads input from one port
\nand sends it to another and at the same time reads output from that port
\nand sends it back to the first port. Ssh uses port 22. If I were to
\ncreate a virtual port (7777) and use port forwarding to connect bill with
\nbob (7777 <-> 22), then I could ssh from my machine as if bob were
\ndirectly accessible from my machine.\n<\/p>\n\n
\nssh -L 7777:bob:22 hamzy@bill
\n<\/code>\n<\/p><\/blockquote>\n\nand then from my machine\n<\/p>\n
\n
\nssh -p 7777 hamzy@localhost
\n<\/code>\n<\/p><\/blockquote>\n\nIt is still easy, right? Unfortunately, it wasn’t. The second problem
\nwas the my connection is unreliable. Whatever the cause, some machine
\nalong the chain from my computer to the remote computer had some temporary
\nproblem and I would loose the connection. Whatever work I was in the
\nmiddle of performing would be lost and I would start over. To solve this
\nI turned to another powerful unix program called screen. Screen
\nessentially allows one session to controll many children session. While
\ndoing this, screen has the ability to run in the background and not mind
\nif you exit the session (logout) which would normally finish the session.
\nSo, on my local machine, I would normally start screen as follows\n<\/p>\n\n
\nscreen
\n<\/code>\n<\/p><\/blockquote>\n\nSsh allows any program to be run instead of the default login program by
\nperforming\n<\/p>\n\n
\nssh -p 7777 hamzy@localhost screen
\n<\/code>\n<\/p><\/blockquote>\n\nEvery time you start screen a new screen process is created. To access a
\npreviously running screen process, I would need to provide some command
\nline arguments\n<\/p>\n\n
\nscreen -DR
\n<\/code>\n<\/p><\/blockquote>\n\nThis will tell screen to detach the first screen process it finds and
\nreattach to it. All I have to do is add a -t option to ssh to provide a terminal for screen to use. So, now I will not loose my work if the connection is
\nlost. All I need to do is put the commands into a loop that sleeps for a
\nlittle bit (to be nice) and retry the connection process. So, was
\neverything solved? No. When I would go back to my session and press a
\nkey, the program would wake up and realize that the connection was lost.
\nAfter waiting for a little while, I would automatically reconnect. I
\ncould improve this. The version that I was using was an old version
\nof ssh. When I used the latest version, I found that I could create a
\nfile called ~\/.ssh\/config and add the following:\n<\/p>\n\n
\nServerAliveInterval 15
\n<\/code>
\nServerAliveCountMax 4
\n<\/code>
\nTCPKeepAlive yes
\n<\/code>\n<\/p><\/blockquote>\n\nThis would tell ssh to ping the remote machine every 15 seconds and
\nterminate the connection if the remote machine did not respond after four
\nattempts. The connection would be lost and my loop would reconnect.\n<\/p>\n\nThis automation runs automatically but everytime I login to a machine it
\nwants a username and a password. I provide the username on the command
\nline since I do not care who knows that fact, but I do not want to provide
\nthe password on the command line. How can I solve this? Ssh has the
\nability to encrypt a password and if I do not want to be asked for a
\npassword, then I can tell it to encrypt an empty password. I do this by
\nthe following:\n<\/p>\n\n
\ncd ~\/.ssh
\n<\/code>
\nssh-keygen -t dsa
\n<\/code>\n<\/p><\/blockquote>\n\nSsh will ask me what password I want and encrypt it with the DSA
\nalgorithm. It creates a plain text file called ~\/.ssh\/id_dsa.pub . I can
\nnotify ssh of this on the remote machine, bob, by putting the contents of
\nthis file into another file called ~\/.ssh\/authorized_keys . This is of
\ncourse not as secure as asking for a password every time. But the
\ntradeoff is it will allow you to automate the connection.\n<\/p>\n\nNow, I can run commands on the remote machine and port the program. To
\ntest the program, I need to run it locally. I will build the program on
\nthe remote machine and copy the data to my local machine. This is called
\nmirroring. How can I copy the files? I could run tar on bob to bundle
\nfiles up and copy that bundle to a tar running on localhost that would
\nunbundle those files. The steps to accomplish this is 1) run tar to
\ncreate a bundle, 2) copy the bundle, and 3) run tar to unbundle the files.
\nThis is too many separate steps. I can improve this.\n<\/p>\n\nUnix allows you to chain programs together by taking program A’s output
\nand forwarding it to program B. Ssh follows this philosophy. So, using
\nssh, I can connect task 1 and task 3 together without performing task 2.
\nNow, I do the following:\n<\/p>\n\n
\ncd location_of_files_on_localhost
\n<\/code>
\nssh -p 7777 hamzy@localhost \"(cd location_of_files_on_bob; tar c *)\" | tar xv
\n<\/code>\n<\/p><\/blockquote>\n\nHowever, there are more efficient ways to mirror files. If the data has
\nnot changed, then nothing needs to be copied. Also, to speed the copying
\nprocess, if only small differences exist between two files, then only
\nthose differences need to be copied. The program that does this in Unix
\nis rsync. The people who wrote rsync knew about ssh. Connecting the
\ndots, I would do the following:\n<\/p>\n\n
\nrsync --rsh=\"ssh -l hamzy -p 7777\" localhost:location_of_files_on_bob location_of_files_on_localhost
\n<\/code>\n<\/p><\/blockquote>\n\nBut what if I only wanted to copy files? I first need to set up the port
\nforwarding but that creates a login session. This session needs to be
\nclosed manually by logging out. After I copy the files, I would leave
\nthis session running. This is not very clean. To solve this I could
\ninstead run some other program which would end automatically. I chose to
\nrun sleep which runs for some amount of time which I arbitrarly set as
\nlong enough (one day).\n<\/p>\n\n
\nssh -L 7777:bob:22 hamzy@bill \"sleep 1d\"
\n<\/code>
\n# run rsync
\n<\/code>\n<\/p><\/blockquote>\n\nOne day is of course not a perfect fit and I am not looking for perfect
\nbut “good enough.” At least, it consumes a minimal amount of resources on
\nbob. And I can actually programmatically find it and kill it (unix
\nterminology).\n<\/p>\n\n
\n# kill the sleep session
\n<\/code>\n<\/p><\/blockquote>\n\nNow, I am happy. I can edit files and copy files. But inspiration hit me
\nand I thought I can use what I just learned and do something else with it.
\nScreen is a pretty powerful program. You can name screen sessions. You
\ncan list running screen sessions. Screen even allows programmatic
\nmanipulation.\n<\/p>\n\nI can start a named screen session that is initially detached:\n<\/p>\n
\n
\nscreen -dm -S AR
\n<\/code>\n<\/p><\/blockquote>\n\nI can check to see if that session is running:\n<\/p>\n
\n
\nscreen -r AR -ls -q
\n<\/code>\n<\/p><\/blockquote>\n\nI can close a named screen session:\n<\/p>\n
\n
\nscreen -dr AR -X quit
\n<\/code>\n<\/p><\/blockquote>\n\nI can add a program Y to a named screen session:\n<\/p>\n
\n
\nscreen -dr AR -X screen Y
\n<\/code>\n<\/p><\/blockquote>\n\nOne of thing that I do, is use a program called bittorrent. Of course,
\nbittorrent is despised by Hollywood since it allows people to digitally
\ncopy media. And I am not going to go into that can of worms. It does
\nhave its legal uses to copy programs that allow themselves to be freely
\ncopied like Linux. This downloading takes a lot of time. So I want to
\nstart the downloading process simply and walk away.\n<\/p>\n\nNow I have all the tools that I need to accomplish that. I can remotely
\ncontrol a machine. I can remotely start a standalone session (screen).
\nAnd I can remotely start bittorrent with screen. So what did I do? I
\nwrote a shell script to do all of that. And how did I start that process?
\nEmail! Huh? Unix people allow email to run programs. When I send a
\nspecially formatted email to a machine, that machine will turn around and
\nrun my script. My script will then start the downloading. Mission
\naccomplished!\n<\/p>\n\nLinks:\n<\/p>\n
\n
- \nhttp:\/\/www.wlug.org.nz\/SSHNotes<\/a>\n<\/li>\n
- \nhttp:\/\/shfs.sourceforge.net\/<\/a>\n<\/li>\n
- \nhttp:\/\/savannah.gnu.org\/projects\/screen\/<\/a>\n<\/li>\n
- \nhttp:\/\/groups-beta.google.com\/group\/comp.unix.shell\/search?group=comp.unix.shell&q=screen&qt_g=1&searchnow=Search+this+group<\/a>\n<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"
For work, I was asked to help port a project to Linux. This involved working on a computer behind a firewall. I am sure that every one is familiar with ssh. For those of you who are not, ssh is a program that encrypts a telnet session. And telnet is a program that allows someone […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[],"class_list":["post-138","post","type-post","status-publish","format-standard","hentry","category-programming"],"_links":{"self":[{"href":"http:\/\/www.hamzy.net\/blog2\/index.php?rest_route=\/wp\/v2\/posts\/138","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.hamzy.net\/blog2\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.hamzy.net\/blog2\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.hamzy.net\/blog2\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/www.hamzy.net\/blog2\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=138"}],"version-history":[{"count":0,"href":"http:\/\/www.hamzy.net\/blog2\/index.php?rest_route=\/wp\/v2\/posts\/138\/revisions"}],"wp:attachment":[{"href":"http:\/\/www.hamzy.net\/blog2\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=138"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.hamzy.net\/blog2\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=138"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.hamzy.net\/blog2\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=138"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}