[CLUE-Tech] Sockets and http question

[Sean LeBlanc]

On 12-29 17:28, David Anselmi wrote:
> Sean LeBlanc wrote:
> >Does anyone have much experience with sockets and http? I'm having some
> >difficulties trying to figure out why my connection just hangs, but only on
> >certain sites.
> >
> >I can include the test code I'm using, if anyone thinks they might be able
> >to help. This is in C++, but it might be more of a http protocol problem.
> 
> You can use ethereal to capture a session from your code and one from a 
> browser to the same site (maybe lynx will make a cleaner capture).  Then 
> you can compare the sessions and see what's different.
> 
> I could look at the code (I'm curious what it does) but I'm no expert on 
> C or C++.  For http, is there a library you can use to do all the 
> protocol stuff?  Perl et. al. have that, it's a shame if you have to 
> write your own.

I'm building ethereal port right now - I tried capturing packets with
ettercap since I already had that, but I wasn't getting the stuff being sent
by Lynx, only what was returning - even though it says it is sniffing both.
I ran this: ettercap -Nzs 192.168.1.3 <remotehostip>, FWIW.

Well, here's the code:

There should be 3 files: server.h, server.cc and test.cc, along with the
Makefile. A "make tst" should make my test program that I've created to
narrow down the problem I've been having. 

Run like this: ./tst www.google.com /
or 
./tst www.allmusic.com /

Both of the above hang, but www.yahoo.com comes across just fine, among
others.

I'm about to try this in Perl, Python or Java, just to see if the problem is
there, too. 


A little background: this code is from MLAC, which I wrote. Much of the code
comes from Spiffy's Audiogalaxy tool - he wrote the code that scrapes
freedb.org and allmusic.com's site for CD data, I just made some changes,
and utilize that data in a different way. I'm starting to think about
re-writing it so that it uses the "CDDB" protocol instead, but I don't want
to go ape just yet if I can make this work. In the long term, that's
definitely a better idea, since websites can (and do) change. 



Makefile:

OBJ             =       main.o server.o stringproc.o \
                        web.o mp3tech.o common.o sort.o
OBJ2            =       test.o server.o
G++             =       g++
CXXFLAGS        =       -O6 -g 

am: $(OBJ)
        $(G++) -o mlac $(OBJ)
tst: $(OBJ2)
        $(G++) -o tst $(OBJ2)
clean:
        rm -f $(OBJ) mlac




####### test.cc########
#include "server.h"

main(int argc, char *argv[])


{
char msg[1024];
//sprintf(msg, "P=amg&uid=SEARCH&sql=%s&opt1=1&x=9&y=2", fixString(artist));

if (argc != 3)
{
 printf("Incorrect usage.\n");
 printf("Usage: %s servername path\n", argv[0]);
 exit(-1);       
}
class server *S = new class server(argv[1], 80);
S->send("GET %s HTTP/1.1\r\n", argv[2]);
//S->send("User-Agent: \r\n", AGENT);
S->send("Host: %s:80\r\n", argv[1]);
S->send("Accept: text/html, text/*\r\n");
S->send("Content-type: application/x-www-form-urlencoded\r\n");
S->send("\r\n\r\n");

while (S->isConnected())
{
  char *sin = S->receive();
  printf("%s", sin);
}
}



############### server.cc ################
include "server.h"

// Using a very large buffer because the code gets really
// confused if the buffer cuts off in the middle of a tag.

#define		BUFSIZE		524288

// ***************** Server class ************************

server::server()
{
	sockfd = -1; sockf = NULL;
}

server::server(char *host, int port)
{
	sockfd = -1; sockf = NULL;
	connectto(host, port);
}

server::~server()
{
	fclose(sockf);
}

void
server::connectto(char *host, int port)
{
	struct sockaddr_in 	peer;
	struct hostent *	hp;
	
	// DNS
	hp = gethostbyname(host);
	if (!hp) return;

	// Create server address data member
	peer.sin_family = AF_INET; peer.sin_port = htons(port);
	bcopy(hp->h_addr_list[0], (char *)&peer.sin_addr, hp->h_length);

	// Create a socket
	sockfd = socket(AF_INET, SOCK_STREAM, 0);

	// Connect socket to server
	if (connect(sockfd, (struct sockaddr *)&peer, sizeof(peer)) == -1)
	{
		// Error in connection
		close(sockfd);
		return;
	} 
	else
	{
		// Make file structure for easier I/O
		sockf = fdopen(sockfd, "a+");
		sockfd = fileno(sockf);
	}
}

int 
server::isConnected()
{
	if (!sockf) return FALSE;
	else return (!feof(sockf));
}

int
server::getSockFD()
{
	return sockfd;
}

void 
server::send(char *fmt, ...)
{
	va_list	ap;
	fd_set	fds;
	char	text[1024];

	bzero(text, 1024);

	va_start(ap, fmt);
	vsprintf(text, fmt, ap);
	va_end(ap);

	FD_ZERO(&fds); FD_SET(sockfd, &fds);
	select(sockfd+1, 0, &fds, 0, 0);
	fprintf(sockf, "%s", text);
}

char *
server::receive()
{
	char ret[BUFSIZE]; bzero(ret, BUFSIZE);

	fgets(ret, BUFSIZE-1, sockf);

	return strdup(ret);
}

int
server::download(char *fn)
{
	char ret[BUFSIZE];

	// Skip the header
	fgets(ret, 1024, sockf);
	while ( (!feof(sockf)) && (strlen(ret) > 2) )
	{
		if (strstr(ret, "404 Object Not Found")) 
			return -1;
		fgets(ret, 1024, sockf);
	}

	FILE *outf = fopen(fn, "w");
	int numread;
	
	numread = fread(ret, 1, BUFSIZE, sockf);
	while (numread > 0)
	{
		fwrite(ret, 1, numread, outf);
		numread = fread(ret, 1, 2048, sockf);
	}

	fclose(outf);

	return 0;
}

		



############## server.h ###########

class server 
{
public:
        server();
        server(char *host, int port);
        ~server();
bash-2.05$ less server.h
#ifndef SERVER_H
#define SERVER_H

#include <stdio.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <string.h>
#include <sys/socket.h>
#include <stdarg.h>
#include <netdb.h>
#include <unistd.h>

#ifndef TRUE
#define TRUE    1
#define FALSE   0
#endif

class server 
{
public:
        server();
        server(char *host, int port);
        ~server();

        void    connectto(char *host, int port);
        int     isConnected();
        int     getSockFD();
        void    send(char *fmt, ...);
        char *  receive();
        int     download(char *fn);
private:
        int     sockfd;
        FILE *  sockf;
};

#endif

-- 
Sean LeBlanc:seanleblanc at americanisp.net  
http://users.americanisp.net/~seanleblanc/
Get MLAC at: http://sourceforge.net/projects/mlac/
Counting in octal is just like counting in decimal -- if you don't use your 
thumbs. 
-Tom Lehrer