Skip to content
Snippets Groups Projects
Commit c6e8a5ed authored by Peder Hovdan Andresen's avatar Peder Hovdan Andresen
Browse files

Initial commit

parents
No related branches found
No related tags found
No related merge requests found
Pipeline #107081 failed with stages
in 48 seconds
*~
# Setting up bookface to work with a gluster backend for images
This guide is in order to set up bookface as a dockerized service,
including memcache.
## Mise en place
This guide assumes the following:
* You have docker swarm ( a single node will work )
* you are using the latest version of bookface
* You also have a cockroach database which is initialized and has a database with tables ready, as described in this guide:
* You have a GlusterFS cluster with a volume, called "images"
## Setting up the GlusterFS volume driver on the swarm
You have to repeat these steps ON ALL nodes in your swarm!
1. Set hostnames with IP's in /etc/hosts. For example:
```
192.168.128.108 gluster-1
192.168.128.241 gluster-2
192.168.128.98 gluster-3
```
2. Install the volume driver plugin:
```
docker plugin install --alias gluster trajano/glusterfs-volume-plugin --grant-all-permissions --disable
docker plugin set gluster SERVERS=gluster-1,gluster-2,gluster-3
docker plugin enable gluster
```
3. Start the bf stack
```
docker stack deploy -c docker-compose-gluster.yml --with-registry-auth bf
```
You should keep track if the services start as desired:
```
docker service ls | grep bf
```
The service is set up to only run one replica of the webserver. This is wise when you want to make sure everything works as intended. If you want more information, try:
```
docker service ps --no-trunc bf_web
```
## Management operations
### Adjusting the number of replicas
Once the service is working properly, you can adjust the number of replicas that are running:
```
docker service update --replicas=3 bf_web
```
### Adjusting the frontpage limit
The default image used by the docker compose file, can adjust the frontpage_limit variable based on an environment variable. There is no need to change the container image and the adjustment can be done dynamically.
In this example, we adjust the frontend_limit to be 100:
```
docker service update --env-add BF_FRONTPAGE_LIMIT=100 --with-registry-auth bf_web
```
FROM ubuntu:18.04
MAINTAINER kyrre@begnum.no
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y apache2 libapache2-mod-php php-mysql git php-memcache curl php-cli php-mbstring unzip php-pgsql glusterfs-client
RUN rm /var/www/html/index.html
RUN mkdir /var/www/html/images
ADD code/* /var/www/html/
ADD config.php /var/www/html
ADD init.sh /
EXPOSE 80
ENTRYPOINT ["/init.sh"]
# Importing data from existing bookfaces
With the latest tag "v7", one can now import data from an existing bookface service. The purpose of this tool is to allow you to migrate to new installations of the service, even if the database has changed.
The current solution is written in PHP and is a part of the new codebase. This means that if you are running a recent container version, then this feature will be available.
When you have done the preparations, you can import the data by calling a special url on the new system.
## Preparation:
The following assumptions are being made:
* You have two bookface systems running. One old and one new.
* The old system is operational and can be accessed via an URL ( does not have to be a floating IP )
* The new system has the database set up with appropriate tables, but contains no data.
In order to protect your new system from others pushing data into yours, you first have to set up a secret key in the new database. This way, only someone knowing the key will be able to initiate migration.
I'm assuming you are running cockroachdb on the new system. Start by opening a console to it. Do this from the docker master:
```
docker run -it --rm --network=roach_cockroachdb cockroachdb/cockroach:v2.1.6 sql --host=cockroachdb-1 --insecure
```
Inside the console, execute the following ( change 'yoursecretkey' with something else ):
```
use bf;
CREATE table config ( key STRING(100), value STRING(500) );
insert into config ( key, value ) values ( 'migration_key', 'yoursecretkey' );
```
You can now disconnect from the console.
## Start the migration
The migration can be initiated from anywhere where you can reach the new system, so it does not have to be publicly available while you move the data over. Start the migration by calling a particular URL, preferrably from a commamnd-line and not in a browser.
Also, this would be a good time to start a screen session and continue from inside it. You do not want to loose connectivity while this is ongoing.
Assuming your new systems IP is 192.168.128.223 with a port number 30002, and your old systems IP is 192.168.128.33 and the key is as described above, the resulting command would be (we'll also add the time command to track how long it takes):
```
time curl -v 'http://192.168.128.223:30002/import.php?entrypoint=192.168.128.33&key=yoursecretkey'
```
Notice the use of single quotes around the URL in order to avoid confusion due to the "&" symbol used to sepparate variables.
This process will take some time, and you can keep track as you watch the output from the curl command.
At the end of the execution, you would see a text like this:
```
<h1>Added 112 users</h1>
</HTML>
```
## What if the call quits with an error before it has finished?
This happens. The import script is designed to continue where it left of last time, so you should be fine running it again and again until all the data is moved. It would be worse if you execute it several times at the same time. So make sure the process has really finished before you start over. For instance, if you lost connectivity, be sure to re-connect to the screen session first.
## Something went terribly wrong and I need to start over
Sure. The original data is still there, so just import it again. But first:
* Clear the data from the databse, by going back to the cockroachdb console and running:
```
delete from users where userid >= 1; delete from posts where postid >= 1; delete from comments where commentid >= 1;
```
* Remove the images in the "images" glusterFS volume ( not strictly needed ).
README.md 0 → 100644
# Welcome to bookface, a fake social media site for learning about web architectures
This site is meant as an educational tool for learning to set up and manage 'big web' applications. The code is not meant to run in production. It is meant to let aspiring sysadmins experience several fault-scenarios so they can improve their skills.
There are many different ways bookface can be deployed. This tutorial
goes through a simple setup using the two core components: a webserver
and a database. You can do everything on a single host or on sepparate
hosts, where one is a webserver and the other the database. The steps
for the webserver can be repeated on multiple webservers who are
behind a load-balancer, but we do not cover the steps of setting up
the load-balancer itself here.
**We assume that Ubuntu 18.04 is used as a base system.**
# Setting up the database (CockroachDB)
Bookface assumes a database within the Postgres family. In this case
we'll use CockroachDB, which is a modern database designed for large
deployments. One nice feature with this database, is that it comes
with a useful web-based status dashboard, which is quite useful when
you want to learn more about the life of a database.
CockroachDB actually assumes that you want to start with
a cluster of three nodes, but we'll settle for only one for now.
All the instructions assume you are root on the machines in question.
## Downloading and installing CockroachDB
You can find the official installation instructions here:
https://www.cockroachlabs.com/docs/stable/install-cockroachdb-linux.html
Our instructions are adapted from theirs. At the time of writing, the latest version of the database is 19.2.2.
Be aware that the version number may have changed, in which case the
URL below needs to be changed as well.
```
wget https://binaries.cockroachdb.com/cockroach-v20.2.4.linux-amd64.tgz
tar xzf cockroach-v20.2.4.linux-amd64.tgz
cp cockroach-v20.2.4.linux-amd64/cockroach /usr/local/bin
```
The three above commands download the latest release as a compressed
archive. The tar command will extract the archive and finally we copy
the binary into the folder '/usr/local/bin', which enables us to
execute the binary as a normal command from the shell.
You can verify these steps by executing the following command:
```
cockroach version
```
If you got a summary of the current version and so on, you are good to
go to the next step.
## Starting the database
Until now, we have only installed the datase, but we are not running
it. Before we start it, let's think about where all the data should be
stored. All persistent databases need somewhere to store all their
stuff. Let's create a folder for it, called `/bfdata` which will
reside at the root of the filesystem. You are free to use whatever
folder you want, just make sure that the root user (or whatever other
user you wish to use to run the database) has read/write access to it.
```
mkdir /bfdata
```
Next, let's start the database:
```
cockroach start --insecure --store=/bfdata --listen-addr=0.0.0.0:26257 --http-addr=0.0.0.0:8080 --background --join=localhost:26257
```
You can see that the `/bfdata` folder is present, which is good. In
addition, you see that we specify to addresses that the database will
listen to. The first one is `0.0.0.0:26257` and means that port 26257
will be available on all network interfaces and that is the port
every database client will try to reach. The other one is for the
dasbboard, which is specified to be available on port 8080.
There are some warnings which are displayd, but that is fine for us
right now. Please remember, that this tutorial assumes that you are in a safe lab
environment. We are obviously taking security-shortcuts here in order to get to
the core of the learning experience.
The next step is to initialize the database:
```
cockroach init --insecure --host=localhost:26257
```
Once this command has finished, we should be able to see wether the
database is actually running in the background or not. Here are to
trusty companions we can use:
```
ps aux | grep cockroach
netstat -anltp | grep cockroach
```
You should be able to see both the running process and that the ports
are actually opened and being listened on.
## Initializing the database
Now that the database is running, we need to log in and create the
user, tables and permissions. The easiest way to access the database,
is to use the cockroach command itself and open up a session:
```
cockroach sql --insecure --host=localhost:26257
```
Notice that your command prompt has changed now that you have started
a command console inside the database. Next, you need to run the
following commands inside the console.
In these three commands, we create the database and define a user. There is no password here, so choose a different username if you want to avoid that others accidently log into your database.
```
CREATE DATABASE bf;
CREATE USER bfuser;
GRANT ALL ON DATABASE bf TO bfuser;
```
These commands will set up the tables:
```
USE bf;
CREATE table users ( userID INT PRIMARY KEY DEFAULT unique_rowid(), name STRING(50), picture STRING(300), status STRING(10), posts INT, comments INT, lastPostDate TIMESTAMP DEFAULT NOW(), createDate TIMESTAMP DEFAULT NOW());
CREATE table posts ( postID INT PRIMARY KEY DEFAULT unique_rowid(), userID INT, text STRING(300), name STRING(150), postDate TIMESTAMP DEFAULT NOW());
CREATE table comments ( commentID INT PRIMARY KEY DEFAULT unique_rowid(), postID INT, userID INT, text STRING(300), postDate TIMESTAMP DEFAULT NOW());
CREATE table pictures ( pictureID STRING(300), picture BYTES );
```
You can end the session by typing `exit` and hitting enter.
## Viewing the CockroachDB dashboard
As mentioned earlier, an attractive feature of CockroachDB is its
web-based dashboard. This provides valuable insight into how the
database sees the world. We can see what queries are executed and how
long they take.
The dashboard is available on port 8080, since that was the port
chosen above. However, since the database does not (and should not)
have a floating IP, it is not available from the outside. One
therefore needs to use a tool such as FoxyProxy to browse the dashboard.
# Setting up a webserver
In this sections, we'll set up a webserver, which will hold the actual PHP code.
On the webserver host, install apache2 and the neccesary dependancies:
```
apt-get update
apt-get install apache2 libapache2-mod-php php-pgsql
```
You should be able to see that the apache2 webserver is running using our two trusty commands `ps` and `netstat`. Apache listens on port 80. You can also browse the IP address of the webserver in order to verify that you can see a standard apache welcome page.
Make sure you have git installed. This can be done with `apt-get install git`, but it is normally present.
The code is downloaded from a git repository like this:
```
git clone https://git.cs.oslomet.no/kyrre.begnum/bookface.git
cd bookface
```
Now, remove the default index.html file and move the main PHP files into apache's document root:
```
rm /var/www/html/index.html
cp code/* /var/www/html/
```
One important part of how bookface works, is it's configuration file. This is where you will specify things like where the database is and the public URL will be. The configuration file is read every time a page is requested from a client and this means you can make dynamic changes to this file in order to change the behaviour of the site without restarting anything. However, it also means that an error in this file will cripple the site. It may be a good idea to come up with a scheme to make sure you have backup and some sort of oversight over this file, especially if you have many webservers.
Next, make a copy of the example configuration file. You may want to keep this file sepparate from the git repository, since it can work with several versions of the code.
```
cp config_example.php /var/www/html/config.php
```
Now, you can edit the config.php file and fill out the variables.
Let's assume that the IP address of the database is 192.168.131.23 and we chose bfuser as a username.
Let's also assume that the floating/public IP address (assuming you are in a cloud environment) is 10.20.0.43.
Let's also assume that the port number of the database is 26257. The config.php file would then look like this:
```
<?php
$dbhost = "192.168.131.23";
$dbport = "26257";
$db = "bf";
$dbuser = "bfuser";
$dbpassw = '';
$webhost = '10.20.0.43';
$weburl = 'http://' . $webhost ;
?>
```
## Test the setup
At this point, you are finally ready to test the entire setup. If this works, the next step will be to populate it with a few users.
There are two ways to test that everything works, on the command-line
and in a web-browser. On the command-line, we would use something like
curl or wget to see if we get a HTML output which makes sense.
```
curl http://floating-ip
```
The information to look for, is this part:
```
<table>
<tr><td>Users: </td><td>0</td></tr>
<tr><td>Posts: </td><td>0</td></tr>
<tr><td>Comments: </td><td>0</td></tr>
</table>
```
If you can see the zeroes, it means we have a connection to the
database and the site is working as intended.
Likewise, in a web-browser you should se a relatively bare but
functioning website. Again, the number og users, posts and comments
should be zero.
If you got to this point, then you are done with your basic setup and
are ready for accepting users!
# How to remove users without images
There are several reasons why some unlucky users don't get any images.
Instead of deleting the entire database it may be better to simply
remove the particular users in question. However, this also means
removing there comments as well as their posts and the comments
on their posts from other users.
First, go to your database and start the console:
```
cockroach sql --insecure --host=localhost:26257
```
Inside the console run each of these commands (each of these is a
single long line):
```
delete from comments where exists (select postid from posts where postid = comments.postid and exists ( select * from users where posts.userid = users.userid and not exists ( select * from pictures where pictureid = users.picture)));
delete from comments where exists ( select * from users where comments.userid = users.userid and not exists ( select * from pictures where pictureid = users.picture));
delete from posts where exists ( select * from users where posts.userid = users.userid and not exists ( select * from pictures where pictureid = users.picture));
delete from users where not exists ( select * from pictures where pictureid = users.picture);
```
# Welcome to bookface, a social website for the 15th century
This site is meant as an educational tool for learning to set up and manage 'big web' applications. The code is not meant to run in production. It is meant to let aspiring sysadmins experience several fault-scenarios so they can improve their skills.
# Installing dependencies (Assumes Ubuntu 14.04)
As a minimum requirement, bookface needs a MySQL database and a webserver with PHP installed as well as the proper libraries for PHP to communicate with the database. The database can be sepparated from the web server and you can have several webservers connected through a loadbalancer.
All the instructions assume you are root on the machines in question.
## Setting up the database
First, install the MySQL database (or MariaDB).
```
apt-get update
apt-get install mysql-server
```
Log into the database and set up the database with the correct permissions.
```
mysql -u root -p
```
Inside the database, create the databse and set the permissions:
```
create database bookface;
grant SELECT,UPDATE,INSERT,CREATE on bookface.* to bookfaceuser@127.0.0.1 identified by 'somepassword';
```
NOTICE: The mysql commads assume that the webserer resides on the same system as the databse. If you have multiple webservers you either have to use a wildcard for the location of the user or add one grant per IP address.
You should verify that the conenction works manually using the mysql client with the credentials you defined.
## Installing apache2
On the webserver hosts, install apache2 and the neccesary dependancies:
```
apt-get install apache2 libapache2-mod-php php-mysql
```
# Bookface installation instructions
Make sure you have git installed. This can be done with ```apt-get install git```
## Downloading the PHP files
```
git clone https://git.cs.hioa.no/kyrre.begnum/bookface.git
cd bookface
```
## Copy the files
Remove the default index.html file and move the main PHP files into apache's document root:
```
cp code/* /var/www/html/
rm /var/www/html/index.html
```
Next, make a copy of the example configuration file. You may want to keep this file sepparate from the git repository, since it can work with several versions of the code.
```
cp config_example.php config.php
```
Now, you can edit the config.php file and fill out the variables. If you are using a load balancer or otherwise external IP for your bookface, make sure to use that in your $webhost variable. When the config.php file is complete, copy it to the other PHP files:
```
cp config.php /var/www/html/
```
The last step is to initiate the tables. You can do so by calling the url:
```
curl http://localhost/createdb.php
```
<html>
<?php
$user = $_GET['user'];
$post = $_GET['comment'];
$postID = $_GET['postID'];
include_once "config.php";
try {
$dbh = new PDO('pgsql:host=' . $dbhost . ";port=" . $dbport . ";dbname=" . $db . ';sslmode=disable',$dbuser, null, array(PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION, PDO::ATTR_EMULATE_PREPARES => true,));
echo "Connection successful!\n<br>";
if ( isset($_GET['nomemcache'])) {
$memcache_override = $_GET['nomemcache'];
}
if ( $memcache_enabled == 1 and ! $memcache_override ){
echo "<! Memcache is enabled !>";
$memcache = new Memcache();
$memcache->addServer ( $memcache_server,"11211" );
}
$result = $dbh->query("insert into comments (text,userid,postid ) values('$post','$user','$postID' );");
if ( isset($memcache) and $memcache ){
$key = "comments_on_$postID";
$memcache->delete($key);
}
} catch (Exception $e) {
echo $e->getMessage() . "\r\n";
}
?>
</html>
<html>
<?php
$user = $_GET['user'];
$post = $_GET['post'];
include_once "config.php";
try {
$dbh = new PDO('pgsql:host=' . $dbhost . ";port=" . $dbport . ";dbname=" . $db . ';sslmode=disable',$dbuser, null, array(PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION, PDO::ATTR_EMULATE_PREPARES => true,));
if ( isset( $_GET['nomemcache']) ){
$memcache_override = $_GET['nomemcache'];
}
if ( isset($memcache_enabled) and $memcache_enabled == 1 and ! $memcache_override ){
echo "<! Memcache is enabled !>";
$memcache = new Memcache();
$memcache->addServer ( $memcache_server,"11211" );
}
$stmt = $dbh->query("select posts from users where userid = $user");
$res = $stmt->fetch(PDO::FETCH_ASSOC);
$posts = $res['posts'] + 1;
$result = $dbh->query("update users set posts = $posts, lastpostdate = now() where userID = $user" );
$result = $dbh->query("insert into posts (text,userid) values('$post','$user');");
if ( isset($memcache_enabled) and $memcache_enabled == 1 and $memcache ){
$memcache->delete("user_list_for_front_page");
$memcache->delete("posts_by_$user");
}
} catch (Exception $e) {
echo $e->getMessage() . "\r\n";
}
?>
</html>
<?php
$dbhost = "192.168.131.45";
$dbport = "26257";
$db = "bf";
$dbuser = "bfuser";
$dbpassw = '';
$webhost = '10.212.139.61';
$weburl = 'http://' . $webhost ;
?>
\ No newline at end of file
<html>
<HEAD>
<LINK href="stylesheet.css" rel="stylesheet" type="text/css">
</HEAD>
<?php
$starttime = time();
echo "\n<table class=headertable>\n<tr>";
echo "<td class=header ><td class=header>";
echo "<h1 class=header><a class=title href='$weburl'>bookface</a></h1>";
echo "</tr></table>\n";
include_once "config.php";
$link = mysql_connect("$dbhost:$dbport", $dbuser, $dbpassw);
if ($link){
$bfdb = mysql_select_db($db,$link);
if ( !$bfdb ){
echo "Cannot use $db: " . mysql_error() ."<br>";
} else {
$result = mysql_query("show tables;");
$row = mysql_fetch_row($result);
if ( $row ){
echo "Tables are created, proceed to index.php<br>\n";
} else {
echo "No tables found, creating...\n";
mysql_query("create table user ( userID INT(10) NOT NULL AUTO_INCREMENT, name VARCHAR(100) , picture BLOB, status VARCHAR(500), posts INT NOT NULL, comments INT NOT NULL, lastPostDate datetime, createDate timestamp default now(), UNIQUE (userID));");
mysql_query("create table posts (postID INT(10) NOT NULL AUTO_INCREMENT, userID INT, text VARCHAR(1000), postDate timestamp default now(), UNIQUE (postID));");
mysql_query("create table comments ( commentID INT(10) NOT NULL AUTO_INCREMENT, postID INT, userID INT, text VARCHAR(500), postDate timestamp default now(), UNIQUE (commentID));");
}
}
}
?>
</HTML>
<HTML>
<?PHP
function generateRandomString($length = 30) {
$characters = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ';
$charactersLength = strlen($characters);
$randomString = '';
for ($i = 0; $i < $length; $i++) {
$randomString .= $characters[rand(0, $charactersLength - 1)];
}
return $randomString;
}
error_reporting(E_ALL);
$oldip = $_GET['entrypoint'];
$migration_key = $_GET['key'];
include_once "config.php";
// $link = mysqli_connect("$olddb:$oldport", $olduser,$oldpassword );
// if ($link){
// $bfdb = mysqli_select_db($link,$olddbname);
// } else {
// echo "Cannot use $db: " . mysqli_error($link) ."<br>";
// }
try {
$dbh = new PDO('pgsql:host=' . $dbhost . ";port=" . $dbport . ";dbname=" . $db . ';sslmode=disable',$dbuser, null, array(PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION, PDO::ATTR_EMULATE_PREPARES => true,));
try {
$stmt = $dbh->query('select value from config where key = \'migration_key\';');
$row = $stmt->fetch(PDO::FETCH_ASSOC);
$config_key = $row['value'];
if ( ! $config_key ){
echo "This system is not set up to import data. You have to specify a migration key first\n";
}
if ( ! $config_key == $migration_key ) {
echo "The migration key does not match\n";
exit(1);
}
} catch (Exception $e) {
echo $e->getMessage() . "\r\n";
exit(1);
}
$stmt = $dbh->query('select max(userid) from users;');
$row = $stmt->fetch(PDO::FETCH_ASSOC);
$max = $row['max'];
$oldpage = file("http://" . $oldip . "/showallusers.php");
$counter = 0;
$dbcounter = 0;
foreach ( $oldpage as $line ){
if ( preg_match("/user=(\d+)\'>([^<]+)/",$line,$result )){
echo "found user: " . $result[2] . " <br>\n";
echo "User ID: " . $result[1] . " <br>\n";
if ( $result[1] <= $max ) {
continue;
}
$imageurl = "http://" . $oldip . "/showimage.php?user=" . $result[1];
echo "Image: " . $imageurl . "<br>\n";
# attemp to insert user
$imagestring = generateRandomString() . ".jpg";
# imagejpeg($img, "images/file.jpg");
# $img = file_get_contents($image);
file_put_contents("images/" . $imagestring, file_get_contents($imageurl));
$showuser = file("http://" . $oldip . "/showuser.php?user=" . $result[1]);
$startdate = "";
$posts = "";
$lastpost = null;
$lastpostid = 0;
foreach ( $showuser as $shline ){
if ( preg_match("/Member since: (.*) posts: (\d+)</",$shline,$shresult )){
$startdate = $shresult[1];
$posts = $shresult[2];
} else if ( preg_match("/postID:(\d+) .*post>(.*)<.td>.*postcontent>(.*)<.td><tr>/",$shline,$shresult )){
# $postresult = $dbh->query("update users set posts = $posts, lastpostdate = now() where userID = $user" );
$postquery = "insert into posts (postid,text,userid,postdate) values('" . $shresult[1] . "','" . $shresult[3] . "','" . $result[1] . "','" . $shresult[2] . "');";
# echo "Postquery: " . $postquery . "\n";
try {
$postresult = $dbh->query($postquery);
} catch (Exception $e) {
echo $e->getMessage() . "\r\n";
}
$lastpostid = $shresult[1];
} else if ( preg_match("/commentpost /",$shline,$shresult )){
# echo "Found comment: " . $shline . "\n";
$commentarray = explode("</tr>", $shline, 10000);
foreach ( $commentarray as $comment ){
# echo "single comment: " . $comment . "\n";
if ( preg_match("/.*commentpost >(.*:\d\d).*user=(\d+)'.*<\/td><td>(.*)<\/td>/",$comment,$cresult)){
echo "Comment: " . $cresult[3]. "\n";
try {
$postresult = $dbh->query("insert into comments (postid,userid,postdate,text) values('" . $lastpostid . "','" . $cresult[2] . "','" . $cresult[1] . "','" . $cresult[3] . "');");
} catch (Exception $e) {
echo $e->getMessage() . "\r\n";
}
}
}
}
}
$query = "insert into users (userid,name,picture,status,posts,comments,createdate,lastpostdate) values(:userid,:username,:image,'',:posts,0,:createdate,:lastpostdate );";
$stmt = $dbh->prepare($query);
$username = substr($result[2],0,49);
$stmt->execute([
':userid' => $result[1],
':posts' => $posts,
':createdate' => $startdate,
':username' => $username,
':image' => $imagestring,
':lastpostdate' => $lastpost
]);
$counter++;
$dbcounter++;
if ( $dbcounter > 100 ){
$dbh = null;
$dbh = new PDO('pgsql:host=' . $dbhost . ";port=" . $dbport . ";dbname=" . $db . ';sslmode=disable',$dbuser, null, array(PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION, PDO::ATTR_EMULATE_PREPARES => true,));
$dbcounter = 0;
}
}
}
echo "<h1>Added " . $counter . " users</h1>\n";
# get list of users ( everything but the picture )
# userID,name,picture,status,posts,comments,lastPostDate,createDate
// $sql = "select userID,name,status,posts,comments,lastPostDate from user order by userID desc";
// $res = mysqli_query($link, $sql);
// while($rec = mysqli_fetch_assoc($res)){
// echo "Found user: " . $res['userID'] . "\n";
// exit(0);
// # prepare the URL for the image
// # Try to insert it with existing ID
// }
# $img = imagecreatefromjpeg($image);
} catch (Exception $e) {
echo $e->getMessage() . "\r\n";
}
#$link = mysql_connect("$dbhost:$dbport", $dbuser,$dbpassw );
# if ($link){
# echo "Connection successful!\n<br>";
# $bfdb = mysql_select_db($db,$link);
# if ( !$bfdb ){
# echo "Cannot use $db: " . mysql_error() ."<br>";
# } else {
# echo "Correct database found<br>\n";
#$img = mysql_real_escape_string(file_get_contents($image));
# echo "$img</br>";
# $fileimg = mysql_real_escape_string(file_get_contents('plot.png'));
# $result = mysql_query(
# echo "Result: " . mysql_error() . "<br>\n";
# if ( ! mysql_error()){
# echo "OK";
# } else {
# mysql_error();
# }
# }
# }
?>
</HTML>
<html>
<HEAD>
<LINK href="stylesheet.css" rel="stylesheet" type="text/css">
</HEAD>
<!-- bookface version 14 -->
<?php
$starttime = time();
$use_file_store_for_images = 0;
include_once "config.php";
echo "\n<table class=headertable>\n<tr>";
echo "<td class=header ><td class=header>";
echo "<h1 class=header><a class=title href='/index.php'>bookface</a></h1>";
echo "</tr></table>\n";
if(isset($_GET['use_file_store_for_images']) or (isset($use_local_images) and $use_local_images)){
$use_file_store_for_images = 1;
}
if ( isset($replica_dbhost) ){
$dbhost = $replica_dbhost;
}
try {
if ( isset($dbpassw) ){
$dbh = new PDO('pgsql:host=' . $dbhost . ";port=" . $dbport . ";dbname=" . $db . ';sslmode=disable',$dbuser, $dbpassw, array(PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION, PDO::ATTR_EMULATE_PREPARES => true,));
} else {
$dbh = new PDO('pgsql:host=' . $dbhost . ";port=" . $dbport . ";dbname=" . $db . ';sslmode=disable',$dbuser, null, array(PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION, PDO::ATTR_EMULATE_PREPARES => true,));
}
$memcache_override = 0;
if ( isset($_GET['nomemcache'])){
$memcache_override = $_GET['nomemcache'];
}
if ( isset($memcache_enabled) and $memcache_enabled == 1 and ! $memcache_override ){
echo "<! Memcache is enabled !>";
$memcache = new Memcache();
$memcache->addServer ( $memcache_server,"11211" );
}
$memcache_for_random = 1;
$memcache_for_counters = 1;
# echo "Correct database found<br>\n";
### Users
if ( isset($memcache_enabled) and $memcache_enabled == 1 and $memcache_for_random ){
echo "<! using memcache for randomized users !>\n";
$random_user = $memcache->get("random_user_id");
if ( $random_user < 1){
echo "<! Cache miss. Going to the DB !>\n";
$stmt = $dbh->query('select userID from users order by random() limit 1;');
$row = $stmt->fetch(PDO::FETCH_ASSOC);
$random_user = $row['userid'];
$memcache->set("random_user_id", $random_user,0,30);
}
echo "<! Random poster: " . $random_user . " !>\n";
} else {
$stmt = $dbh->query('select userID from users order by random() limit 1;');
$row = $stmt->fetch(PDO::FETCH_ASSOC);
echo "<! Random poster: " . $row['userid'] . " !>\n";
}
### POST and USER
if ( isset($memcache_enabled) and $memcache_enabled == 1 and $memcache_for_random ){
echo "<! using memcache for randomized posters !>\n";
$random_userid = $memcache->get("random_userid");
$random_postid = $memcache->get("random_postid");
if ( $random_userid < 1){
echo "<! Cache miss. Going to the DB !>\n";
$stmt = $dbh->query('select postid,userid from posts order by random() limit 1;');
$row = $stmt->fetch(PDO::FETCH_ASSOC);
$random_userid = $row['userid'];
$random_postid = $row['postid'];
$memcache->set("random_userid", $random_userid,0,60);
$memcache->set("random_postid", $random_postid,0,60);
}
echo "<! Random post: " . $random_postid . " !>\n";
echo "<! Random user: " . $random_userid . " !>\n";
} else {
$stmt = $dbh->query('select postid,userid from posts order by random() limit 1;');
$row = $stmt->fetch(PDO::FETCH_ASSOC);
echo "<! Random post: " . $row['postid'] . " !>\n";
echo "<! Random user: " . $row['userid'] . " !>\n";
}
if ( isset($memcache_enabled) and $memcache_enabled == 1 and $memcache_for_counters ){
$user_count = $memcache->get("user_count");
$posts_count = $memcache->get("posts_count");
$comments_count = $memcache->get("comments_count");
echo "<! Using memcache to display counters !>\n";
if ( $user_count < 1 ){
echo "<! Count too low or Cache miss. Going to the DB !>\n";
$stmt = $dbh->query('select count(userID) from users;');
$row = $stmt->fetch(PDO::FETCH_ASSOC);
$user_count = $row['count'];
$stmt = $dbh->query('select count(postID) from posts;');
$row = $stmt->fetch(PDO::FETCH_ASSOC);
$posts_count = $row['count'];
$stmt = $dbh->query('select count(commentID) from comments;');
$row = $stmt->fetch(PDO::FETCH_ASSOC);
$comments_count = $row['count'];
$user_count = $memcache->set("user_count",$user_count,0,180);
$posts_count = $memcache->set("posts_count",$posts_count,0,180);
$comments_count = $memcache->set("comments_count",$comments_count,0,180);
}
echo "<table>\n";
echo "<tr><td>Users: </td><td>" . $user_count . "</td></tr>\n";
echo "<tr><td>Posts: </td><td>" . $posts_count . "</td></tr>\n";
echo "<tr><td>Comments: </td><td>" . $comments_count . "</td></tr>\n";
echo "</table>\n";
} else {
echo "<table>\n";
$stmt = $dbh->query('select count(userID) from users;');
$row = $stmt->fetch(PDO::FETCH_ASSOC);
echo "<tr><td>Users: </td><td>" . $row['count'] . "</td></tr>\n";
$stmt = $dbh->query('select count(postID) from posts;');
$row = $stmt->fetch(PDO::FETCH_ASSOC);
echo "<tr><td>Posts: </td><td>" . $row['count'] . "</td></tr>\n";
$stmt = $dbh->query('select count(commentID) from comments;');
$row = $stmt->fetch(PDO::FETCH_ASSOC);
echo "<tr><td>Comments: </td><td>" . $row['count'] . "</td></tr>\n";
echo "</table>\n";
}
$user_list_for_front_page = array();
echo "<h2>Latest activity</h2>\n";
if ( isset($memcache_enabled) and $memcache_enabled == 1 and $memcache ){
$user_list_for_front_page = $memcache->get("user_list_for_front_page");
}
if ( empty($user_list_for_front_page) ) {
$sql = "select userID,name,status,posts,comments,lastPostDate,picture from users order by lastPostDate desc";
if ( isset($frontpage_limit) ){
$sql = $sql . " limit $frontpage_limit";
}
# $res = $dbh->query($sql);
foreach ($dbh->query($sql) as $rec)
$user_list_for_front_page[] = $rec;
}
// cache for 10 minutes
if ( isset($memcache) and $memcache ){
$memcache->set("user_list_for_front_page", $user_list_for_front_page,0,600);
}
echo "<table class=row >\n";
echo "<tr><td></td><td>Name:</td><td>Posts</td></tr>\n";
$alternator = 0;
foreach ( $user_list_for_front_page as $res ){
$style = "class=row";
if ( $alternator % 2 ){
$style = "";
}
$alternator++;
echo "<tr >\n";
if ( $use_file_store_for_images ){
echo "<td $style ><a href='/showuser.php?user=" . $res['userid']. "'><img src='/images/" . trim($res['picture']) . "'></a></td>";
} else {
echo "<td $style ><a href='/showuser.php?user=" . $res['userid']. "'><img src='/showimage.php?user=" . trim($res['userid']) . "'></a></td>";
}
echo "<td $style ><a href='/showuser.php?user=" . $res['userid']. "'>" . trim($res['name']) . "</a></td>";
echo "<td $style >" . $res['posts'] . "</td>";
echo "</tr></a>\n";
}
echo "</table>\n";
$totaltime = time() - $starttime;
echo "load time: " . $totaltime . "s\n";
} catch (Exception $e) {
echo $e->getMessage() . "\r\n";
}
?>
</html>
<HTML>
<?PHP
function generateRandomString($length = 30) {
$characters = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ';
$charactersLength = strlen($characters);
$randomString = '';
for ($i = 0; $i < $length; $i++) {
$randomString .= $characters[rand(0, $charactersLength - 1)];
}
return $randomString;
}
error_reporting(E_ALL);
$username = $_GET['user'];
$image = $_GET['image'];
$use_file_store_for_images = 0;
echo "Creating user: " . $username . "<br>";
echo "image:" . $image . "<br>\n";
include_once "config.php";
if(isset($use_local_images) and $use_local_images == 1){
$use_file_store_for_images = 1;
}
try {
$dbh = new PDO('pgsql:host=' . $dbhost . ";port=" . $dbport . ";dbname=" . $db . ';sslmode=disable',$dbuser, null, array(PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION, PDO::ATTR_EMULATE_PREPARES => true,));
# $img = imagecreatefromjpeg($image);
$imagestring = generateRandomString() . ".jpg";
# imagejpeg($img, "images/file.jpg");
# $img = file_get_contents($image);
$query = "insert into users (name,picture,status,posts,comments) values(:username,:image,'',0,0 );";
$stmt = $dbh->prepare($query);
$stmt->execute([
':username' => $username,
':image' => $imagestring,
]);
if($use_file_store_for_images){
file_put_contents("images/" . $imagestring, file_get_contents($image));
} else {
$img = pg_escape_bytea(file_get_contents($image));
$image_query = "insert into pictures (pictureID,picture) values('". $imagestring . "','$img');";
# echo "\n " . $image_query;
$stmt = $dbh->prepare($image_query);
$stmt->execute([]);
}
} catch (Exception $e) {
echo $e->getMessage() . "\r\n";
}
#$link = mysql_connect("$dbhost:$dbport", $dbuser,$dbpassw );
# if ($link){
# echo "Connection successful!\n<br>";
# $bfdb = mysql_select_db($db,$link);
# if ( !$bfdb ){
# echo "Cannot use $db: " . mysql_error() ."<br>";
# } else {
# echo "Correct database found<br>\n";
#$img = mysql_real_escape_string(file_get_contents($image));
# echo "$img</br>";
# $fileimg = mysql_real_escape_string(file_get_contents('plot.png'));
# $result = mysql_query(
# echo "Result: " . mysql_error() . "<br>\n";
# if ( ! mysql_error()){
# echo "OK";
# } else {
# mysql_error();
# }
# }
# }
?>
</HTML>
\ No newline at end of file
<html>
<HEAD>
<LINK href="stylesheet.css" rel="stylesheet" type="text/css">
</HEAD>
<?php
include_once "config.php";
try {
$dbh = new PDO('pgsql:host=' . $dbhost . ";port=" . $dbport . ";dbname=" . $db . ';sslmode=disable',$dbuser, null, array(PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION, PDO::ATTR_EMULATE_PREPARES => true,));
// $bfdb = mysql_select_db($db,$link);
# echo "Correct database found<br>\n";
// $result = mysql_query("select userID,name,status,posts,comments from user");
$sql = "select userid,name,status,posts,comments from users";
echo "<table>\n";
foreach ($dbh->query($sql) as $rec){
echo "<tr>\n";
echo "<td><a href='/showuser.php?user=" . $rec['userid']. "'>" . $rec['name'] . "</a></td>";
echo "<td>" . $rec['posts'] . "</td>";
echo "<td><a href='/showuser.php?user=" . $rec['userid']. "'><img src='/showimage.php?user=$rec[userid]'></a></td>";
echo "</tr></a>\n";
}
echo "</table>\n";
} catch (Exception $e) {
echo $e->getMessage() . "\r\n";
}
?>
</html>
<?php
$user = $_GET['user'];
include_once "config.php";
$use_file_store_for_images = 0;
if(isset($_GET['use_file_store_for_images'])){
$use_file_store_for_images = 1;
}
if ( isset($replica_dbhost) ){
$dbhost = $replica_dbhost;
}
try {
$dbh = new PDO('pgsql:host=' . $dbhost . ";port=" . $dbport . ";dbname=" . $db . ';sslmode=disable',$dbuser, null, array(PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION, PDO::ATTR_EMULATE_PREPARES => true,));
$memcache_override = 0;
if ( isset($_GET['nomemcache'])){
$memcache_override = $_GET['nomemcache'];
}
if ( $memcache_enabled == 1 and ! $memcache_override ){
$memcache = new Memcache();
$memcache->addServer ( $memcache_server,"11211" );
}
$key = "picture_of_" . $user;
if ( $memcache ){
$picture_of_user = $memcache->get($key);
}
if( $picture_of_user == false){
$picture_of_user = "";
if( $use_file_store_for_images ){
$picture_of_user = file_get_contents("images/" . $row["picture"]);
} else {
$sql = "select picture from users where userid = $user";
$stmt = $dbh->query($sql);
$row = $stmt->fetch(PDO::FETCH_ASSOC);
$image_query = "select picture from pictures where pictureid = '" . $row["picture"] . "'";
# echo $image_query . "\n";
$stmt = $dbh->query($image_query);
$imagerow = $stmt->fetch(PDO::FETCH_ASSOC);
# print_r($imagerow);
ob_start();
fpassthru($imagerow["picture"]);
$picture_of_user = ob_get_contents();
ob_end_clean();
#echo $picture_of_user;
}
}
// cache for 10 minutes
if ( $memcache ){
$memcache->set($key, $picture_of_user,0,6000);
}
header("Content-type: image/jpg");
echo $picture_of_user;
} catch (Exception $e) {
echo $e->getMessage() . "\r\n";
}
?>
<html>
<HEAD>
<LINK href="stylesheet.css" rel="stylesheet" type="text/css">
</HEAD>
<?php
$user = $_GET['user'];
$use_file_store_for_images = 0;
include_once "config.php";
if(isset($use_local_images) and $use_local_images == 1){
$use_file_store_for_images = 1;
}
echo "\n<table class=headertable>\n<tr>";
echo "<td class=header ><td class=header>";
echo "<h1 class=header><a class=title href='/index.php'>bookface</a></h1>";
echo "</tr></table><br><br>\n";
if ( isset($replica_dbhost) ){
$dbhost = $replica_dbhost;
}
try {
$dbh = new PDO('pgsql:host=' . $dbhost . ";port=" . $dbport . ";dbname=" . $db . ';sslmode=disable',$dbuser, null, array(PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION, PDO::ATTR_EMULATE_PREPARES => true,));
$memcache_override = 0;
if ( isset($_GET['nomemcache'])){
$memcache_override = $_GET['nomemcache'];
}
if ( isset($memcache_enabled) and $memcache_enabled == 1 and ! $memcache_override ){
echo "<! Memcache is enabled !>";
$memcache = new Memcache();
$memcache->addServer ( $memcache_server,"11211" );
}
$stmt = $dbh->query("select name,status,posts,comments,createdate,picture from users where userid = $user");
$res = $stmt->fetch(PDO::FETCH_ASSOC);
echo "\n<table class=headertable>\n<tr>";
if ( $use_file_store_for_images ){
echo "<td class=header ><img src='/images/" . trim($res['picture']) . "'><td class=header>";
} else {
echo "<td class=header ><img src='/showimage.php?user=" . $user . "'><td class=header>";
}
echo "<h2 class=hader>" . trim($res['name']) . "</h2>";
echo "</tr></table>\n";
echo "<b>Member since: " . $res['createdate'] . " posts: " . $res['posts'] . "</b><br>\n";
$posts_by_user = array();
if ( isset($memcache) and $memcache ){
$posts_by_user = $memcache->get("posts_by_$user");
}
if (empty($posts_by_user)){
echo "<! No memcache object found: posts_by_$user !>\n";
$posts_by_user = array();
$sql = "select postid,text,postdate from posts where userid = $user order by postdate desc;";
foreach ($dbh->query($sql) as $rec)
$posts_by_user[] = $rec;
// cache for 10 minutes
if ( $memcache ){
$memcache->set("posts_by_$user", $posts_by_user,0,600);
}
}
$postcount = 0;
$table = "<table>\n";
if ( isset($posts_by_user) ){
foreach ( $posts_by_user as $res ){
$table .= "<! postID:". $res['postid'] . " !><tr><td class=post>" . $res['postdate'] . "</td><td class=postcontent>" . $res['text'] . "</td><tr>\n";
$postcount++;
$key = "comments_on_" . $res['postid'];
$comments_on_post = array();
if ( isset($memcache) and $memcache ){
$comments_on_post = $memcache->get($key);
}
if( empty($comments_on_post)){
echo "<! No memcache object for comment found: $key !>\n";
$comments_on_post = array();
$sql = "select commentid,text,userid,postdate from comments where postid = " . $res['postid'] . " order by postdate asc;";
foreach ($dbh->query($sql) as $rec)
$comments_on_post[] = $rec;
// cache for 10 minutes
if ( $memcache ){
$memcache->set($key, $comments_on_post,0,600);
}
}
if ( count($comments_on_post) > 0 ){
$table .= "</table>\n<table class=commentrow>\n";
foreach ( $comments_on_post as $cres ){
$stmt = $dbh->query("select name,picture from users where userid = $cres[userid]");
$ures = $stmt->fetch(PDO::FETCH_ASSOC);
if ( $use_file_store_for_images ){
$table .= "<tr ><td class=commentpost >" . $cres['postdate'] . "</td><td><a href='/showuser.php?user=" . $cres['userid']. "'><img src='/images/".trim($ures['picture']) ."'></a></td><td><b><a href='/showuser.php?user=" . $cres['userid']. "'>" . trim($ures['name']) . ": </a></b></td><td>" . $cres['text'] . "</td></tr>";
} else {
$table .= "<tr ><td class=commentpost >" . $cres['postdate'] . "</td><td><a href='/showuser.php?user=" . $cres['userid']. "'><img src='/showimage.php?user=".trim($cres['userid']) ."'></a></td><td><b><a href='/showuser.php?user=" . $cres['userid']. "'>" . trim($ures['name']) . ": </a></b></td><td>" . $cres['text'] . "</td></tr>";
}
}
$comments_on_post = array();
$table .= "</table>\n<table>\n";
}
}
}
$table .= "</table>\n";
echo "$table\n";
echo "</table>\n";
} catch (Exception $e) {
echo $e->getMessage() . "\r\n";
}
?>
</html>
body {
background-color: white;
font-family: Verdana, Arial, Helvetica, sans-serif;
font-size: 10pt;
padding-top: 5px;
padding-left: 15px;
padding-right: 0px;
padding-bottom: 15px;
margin: 6px;
}
img {
border: 0;
}
td {
background-color: white;
font-family: Verdana, Arial, Helvetica, sans-serif;
font-size: 10pt;
}
td.post {
background-color: lightblue;
font-family: Verdana, Arial, Helvetica, sans-serif;
font-size: 10pt;
width: 150px;
}
td.commentpost {
background-color: #666699;
font-family: Verdana, Arial, Helvetica, sans-serif;
font-size: 10pt;
width: 150px;
}
table.commentrow {
padding-left: 20px;
}
td.comment {
background-color: white;
font-family: Verdana, Arial, Helvetica, sans-serif;
font-size: 10pt;
width: 150px;
}
td.postcontent {
background-color: white;
font-family: Verdana, Arial, Helvetica, sans-serif;
font-size: 10pt;
width: 400px;
}
h1 {
color: #666699;
font-family: Verdana, Arial, Helvetica, sans-serif;
}
h1.header {
color: white;
background-color: #666699;
font-size: 30pt;
font-family: Verdana, Arial, Helvetica, sans-serif;
}
h2.header {
color: white;
background-color: #666699;
font-size: 19pt;
font-family: Verdana, Arial, Helvetica, sans-serif;
}
td.header {
color: white;
background-color: #666699;
font-size: 14pt;
font-family: Verdana, Arial, Helvetica, sans-serif;
padding-right: 40px;
}
table.row {
background-color: grey;
border-collapse: collapse;
width: 50%;
}
td.row {
background-color: #666699;
}
table.headertable {
background-color: #666699;
font-size: 19pt;
font-family: Verdana, Arial, Helvetica, sans-serif;
}
h3, h4, h6
{
color: #666699;
font-family: Verdana, Arial, Helvetica, sans-serif;
text-align: left;
}
a:link {
color: blue;
}
a:link.title {
color: white;
}
a.title {
text-decoration: none;
}
a:visited
{
color: orange;
}
a:visited.title
{
color: white;
}
a:hover {
color: teal;
}
p {
color: #CCCCCC;
font-style: bold;
font-family: Helvetica, Verdana, Arial, sans-serif;
font-size: 10pt;
text-align: left;
}
p.author {
color: grey;
font-style: italic;
font-family: sans-serif, Verdana, Arial, Helvetica;
font-size: 10pt;
padding-left: 20px;
}
\ No newline at end of file
<?php
$use_local_images = 1;
$dbhost = getenv("BF_DB_HOST");
$dbport = getenv("BF_DB_PORT");
$db = getenv("BF_DB_NAME");
$dbuser = getenv("BF_DB_USER");
$dbpassw = getenv("BF_DB_PASS");
$webhost = getenv("BF_WEBHOST");
$imagepath = getenv("BF_IMAGE_PATH");
$weburl = 'http://' . $webhost ;
$frontpage_limit = 100000;
if ( getenv("BF_FRONTPAGE_LIMIT") ){
$frontpage_limit = getenv("BF_FRONTPAGE_LIMIT");
}
if ( getenv("BF_MEMCACHE_SERVER")){
$memcache_enabled_pictures = 1;
$memcache_server = getenv("BF_MEMCACHE_SERVER");
$memcache_enabled = 1;
}
?>
<?php
$dbhost = "";
$dbport = "3306";
$db = "";
$dbuser = "";
$dbpassw = '';
$webhost = '';
$weburl = 'http://' . $webhost ;
?>
version: '3.4'
configs:
bf_config:
file: ./config.php
networks:
webnet:
external:
name: roach_cockroachdb
volumes:
bf_images:
driver: gluster
name: "images"
services:
web:
image: dock01:5000/bf:v4
configs:
- source: bf_config
target: /var/www/html/config.php
ports:
- "80"
networks:
- webnet
environment:
BF_MEMCACHE_SERVER: memcache
BF_FRONTPAGE_LIMIT: 10
volumes:
- bf_images:/var/www/html/images
deploy:
replicas: 1
memcache:
image: memcached
networks:
- webnet
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment