A Software Development Company

How to change the SSD of Mac Book Pro Late 2012

Published: 11/20/2016 9:12:00 AM

I bought my Mac Book Pro with the best parameters at its time - CPU, GPU - Video Card, Maximum allowed RAM Memory 16GB, but the only thing I made a compromise was the SSD. At that time to buy 512 GB instead of mine 256GB would cost me additional £600. I remember the overall price of the laptop was £1700 with student discount (I was a student in the University of Bristol at that time), otherwise would be more than £2000 (can't remember how much exactly). Anyway the compromise was made and after just a couple of months it proofed itself to be very a bad one - too little space - this means nothing except programs and VMWare - Windows 7 (with Microsoft Development Package Visual Studio, SQL Server and so on) and X-Code. It was a struggle. I am a Software Engineer and usually use my laptop only for work but still I wanted to listen to music, edit home videos and other staff.

In 2014 I bought this storage expansion card - SDHC:


from Amazon. That was quick and cheap solution 128GB for 80 USD.
So now I could actually store images, videos and music. But it was slow and not as fast as SSD and for sure not useful for running applications directly from there.

The last option for me was fixing the actual problem - change of SSD. That was difficult choice as it was not officially done by apple.

After a lot of googling around I decided not to do it. This year - 2016 after I sow the new MacBook pro's - the fact that they still have 16 GB RAM, not so much better processors and most importantly the price was close to 3000£ if I wanted better components, I decided I will try to postponethat buy for some years.



So my next step was - go to amazon buy SSD and change it yourself.

It was £450

Then it turns out that is quite an easy process - everything is written as part of the instructions that are coming with the memory. First from system recovery mode copy the OS on the new SSD. Open, Replace and you are ready. It turns out that is exactly this. Do not be afraid!

Some pictures of my mac during the process:

Step1: place it and prepare for change so that each screw can be placed next to it's positions. The screws are 3 types (I am not sure), so not to care place them next to the whole where they have to go to.


Step2: Remove the screws and clean the dust with cotton buds. If you have never open your mac you will find a lot of staff in there.





That was it. After I replaced the SSD I careful put back the screws in their exact positions and all done. It was working fine, although it took some time for the spotlight search to reindex the files - couple of hours. Since then it was running absolutely fine. It is worth testing with Disk Speed Test - software for mac before you change your SSD and after - to compare if the new one is slower or faster. For me (Mac Late 2012) it turns out that there is no difference. Just when you test make sure you have not much running apps - so that the whole RAM can be used to test the SSD.


I know that 2016 Mac's have some unbelievable Read and Write Speed of their SSDs, but till the price is that hight I will be waiting for some years again before making change.

That's my Mac Now - from 256 GB to 1 TB:


Hope you get just a little inspired and go and change the SSD instead of selling what you have and buying a new model for more than £3000.

Comment
Home

My first lego Mindstorm Ev3 console app

Published: 4/9/2016 3:36:00 PM

It was interesting experimenting with the intelligent brick and the software that comes with it. (EV3 PROGRAMMING SOFTWARE (PC/MAC))
Much more interesting is to get a library which allows .net developers to write c# code to "talk" directly with the device.

The library https://github.com/velchev/legoev3 is available in github but also there is a nuget package for that.

Here is my simple first app: https://github.com/velchev/LegoMindstormEv3

Now is an initial setup that moves a servo. I will try to modify it so it executes commands from the command line will see how it goes.

Comment
Home

Bootstrap Modal Dialog - Loading Content from MVC Partial View

Published: 10/11/2014 5:45:00 PM

Original content - my post in code project. From there you can download and the source code.

Introduction

I will show a simple way to display the content of partial view in a bootstrap dialog. I will also show how to fix the dialog doesn't appear issue after 3.1.0 version. There is no need to write additional Ajax calls - this is handled internally by Bootstrap.

Background 

When you have a list - table of data and want to add a record, it's one of the better ideas to use a modal dialog - just because there will be no need to go to another page. Also for the deletion of records - confirmation could be done using bootstrap modal dialogs. There are many other places where you can use modal dialogs.
I saw online many different ways to create modal dialogs... many bad ways like: 
1. Creating all dialogs on the page and then just show or hide them using JavaScript. This means adding partial views to a page - render all the information and then if the user needs to see the dialog - it will appear.
2. Making Ajax calls to display the content. This means for every dialog a call .... not effective. It is better to use a library for that.
The question now is how to do it in a nice and simple way?

  • On the main layout, you need to have a modal container - the place where the dialogs will be loaded.

  • Set some styles - I have put that in the main layout just for the example, but you can put it in the correct place in your project.
  
    </style>   -->

  • Add JavaScript - this will make an Ajax call internally and will inject the partialview content into the modal container.

<script type="text/javascript">
        $(
function () {
           
// Initialize numeric spinner input boxes
            //$(".numeric-spinner").spinedit();
            // Initialize modal dialog
            // attach modal-container bootstrap attributes to links with .modal-link class.
            // when a link is clicked with these attributes, bootstrap will display the href content in a modal dialog.
            $('body').on('click', '.modal-link', function (e) {
                e.preventDefault();
                $(
this).attr('data-target', '#modal-container');
                $(
this).attr('data-toggle', 'modal');
            });
           
// Attach listener to .modal-close-btn's so that when the button is pressed the modal dialog disappears
            $('body').on('click', '.modal-close-btn', function () {
                $(
'#modal-container').modal('hide');
            });
           
//clear modal cache, so that new content can be loaded
            $('#modal-container').on('hidden.bs.modal', function () {
                $(
this).removeData('bs.modal');
            });
            $(
'#CancelModal').on('click', function () {
               
return false;
            });
        });
   
</script>

  • Create a Controller Action, that will return partial view.

 public ActionResult ViewLyubomir()
{
    
return PartialView("_Lyubomir");
 }

  • Create an action that will process the result of the post occurring in the partial view.

[HttpPost]
 
public ActionResult Lyubomir()
 {
    
return RedirectToAction("Index");
 }

  • Create a partial view:
   <div class="modal-body">
    <div class="alert alert-warning">
       
<span class="glyphicon glyphicon-warning-sign"></span>
        Lorem ipsum dolor sit amet, consectetur adipiscing elit.
        Donec consequat nisl a nibh tincidunt condimentum.
        Nullam in augue vitae augue dictum eleifend.
        Nunc porta fermentum metus, quis efficitur lectus scelerisque ac.
        Quisque egestas sit amet nunc in interdum.
        Etiam malesuada maximus nisi, id tempus metus.
        Vivamus at sapien ut metus aliquet sodales sed id magna.
        Integer in auctor ex. Phasellus tempor, est vel placerat dapibus,
        nulla dui tempor ligula, at pulvinar libero nunc gravida metus.
        Proin non feugiat felis. Proin ut ultrices ex. Morbi aliquet lacinia elit at bibendum.
        Nunc facilisis, neque a finibus dignissim, turpis felis vulputate diam,
        a tristique tellus nibh nec nulla. Suspendisse eget augue non turpis
        dignissim euismod eget eget odio. Donec sit amet nibh cursus, efficitur
        nibh in, sodales arcu. Pellentesque pulvinar consequat lacus ac porttitor.
   
</div>
    @using (Html.BeginForm("Lyubomir", "Home", FormMethod.Post))
    {
       
<div class="row">
           
       
</div>
       
<div class="row">
           
<div class="col-md-4 col-md-offset-4">
               
<button type="button" class="btn btn-default"
               
data-dismiss="modal">Cancel</button>
               
<button type="submit" id="approve-btn"
               
class="btn btn-danger">Save</button>
           
</div>
       
</div>
    }
</div>
<script type="text/javascript">
    $(
function () {
        $(
'#approve-btn').click(function () {
            $(
'#modal-container').modal('hide');
        });
    });
</script>
   

  • The last step is to create an action link with specific attributes.
@Html.ActionLink("Lyubomir ", "ViewLyubomir", "Home",
null, new { @class = "modal-link btn btn-success" })

How Does It Work?

On point 7 when the user clicks on the link, based on the attribute - modal-link we have an attached click event for all the links with that attribute - point 3. On point 3, the default action of a link will be prevented and the content to where this link points to will be injected into the model-container div point 1. When the user posts in the modal dialog - point 6, the data will be processed in the corresponding controller - point 5.
When you open the project - Home view has a link button - Lyubomir. Click on that and you will see the magic happening.

Points of Interest

As you can see - nice and simple, no direct Ajax calls to display the content - everything is handled by Bootstrap. Just remember the changing of the versions and how this affects the example. You can search online for that but, it seems a cosmetic issue but it took me quite some time to figure out where to apply it.

Comment
Home

SQL Scripts Runner

Published: 11/6/2013 1:43:00 PM

Introduction


A simple console application, which executes all SQL files added by developers working with decentralised development environment.

For downloading the source code: Codeproject Article

PROBLEM

How do you manage SQL scripts in a decentralised development environment?

The Figure below kind of describes the what I had in mind when I refer to a decentralised development environment?


Figure 1 
Decentralised Development Environment is the one with the local storage - the dashed box
Centralised Development Environment has only centralised data storage. 

Usually there is a main database and every developer has his own copy locally.

This type of development environment allows for independent work without the need of a network connection; if there is an error with the main storage, the others will still be able to work with their local database copies.

However a great disadvantage is that during development each developer has to maintain the main database and it’s local one. This means creating a script constantly with the necessary updates. Another option is to create a backup and restore the main database locally (this would take a lot of time and is not considerate as good practice).

So how can one keep track of the Database changes?
In this article I propose two solutions.
  1. Store the script in a common storage available for all developers – for example RedMine (http://www.redmine.org).
  2. Add all the SQL scripts in the project source control.


This first option separates the DB scripts from the concrete project, but does not provide for versioning. 
The second option however allows for versioning and when the latest version of the SQL scripts have been retrieved from the source control the developer can run it as a console executable manually. (It would be ideal if there were a way to do this automatically, this I leave to the general public.). 

What the executable should do?
  • Should be configurable – this should specify the SQL scripts folder.
  • The executable should run the scripts in order.
  • If there is a syntactic error with the script, the application should display it before even running the script - validation.
  • If there is a runtime error with the script all the scripts should roll back. The file and the line at which the error occurred should be displayed.
  • When there is no error then all the scripts should be executed and the changes applied.
  • Should have a way to test the results – simple Unit Testing.

One very important thing to be mentioned is that the SQL script should be written in a way that it can be executed many times (this means having check conditions or a drop and create statement).

Implementation

Returning a list of SQL files

All the files from a directory and its subdirectories should be returned but in the right order – this means that the files from folder Ver11.0 should be not after Ver1.0 if there is folder with name Ver2.0. This means that the sort order should be based on the values of the versions, but not by the string values.
That’s very easily implemented by a LINQ expression:
foreach (var d in Directory.GetDirectories(dir).OrderBy(x => float.Parse(Path.GetFileName(x).Substring(Configuration.Configuration.Pproperties.ScriptVersionPrefix.Length))))
{

Display the execution percentage


When \r is used the cursor goes back to the beginning of the current line and then can be rewritten. 

Console.Write("\r{0}{1}% complete", message, percent);

How to find SQL compile time errors?


The parsing capabilities of SQL Server Management Studio were used for this. I have used the two assemblies: Microsoft.Data.Schema.ScriptDom; Microsoft.Data.Schema.ScriptDom.Sql;

These two assemblies can be found in: C:\Windows\Microsoft.NET\assembly\GAC_MSIL
If you have installed MS SQL Management Studio
public static bool ParseSqlFile(string file)
{
     IList errors = new List();
     bool hasErros = false;
     using (TextReader reader = File.OpenText(file))
     {
         var parser = new TSql100Parser(true);
         var script = parser.Parse(reader, out errors) as TSqlScript;
         hasErros = errors.Count > 0;
         foreach (var parseError in errors)
         {
              Errors.ProcessErrors(file, parseError);
         }
      }
      return hasErros;
}

Main method of the application

All comes together in the main method. Here after the connection to the database is open, the SQL files are returned in a collection, then for every file its SQL script is parsed and executed as part of transaction so that when the execution is unsuccessful the data can be rolled back.

Please note:
//TODO: remove
Thread.Sleep(500);

This should be removed when it is used in practice. It is good for demo purposes so that the execution percentage can be seen.

The next section is not important for the current project and can be skipped. If after you get the source code you do not understand something about it you can get back to it.

Getting the configuration in OOP way

In “Configuration” folder can be found three files, which are responsible for getting the configuration settings from the App.config in OOP way. A custom attribute defines the key of the setting and after that the attribute is applied to a property, which is set by reflection when the console application starts.

The custom attribute  class:
[AttributeUsage(AttributeTargets.Property, Inherited = false)]
class ConfigAttribute : Attribute
{
     ///

     /// The key
     ///

 private readonly string key;
Class with properties which have attributes of the custom attrubute type
public class ConfigProperties
{
   ///

   /// Gets or sets a value indicating whether [use AU s_ AIS].
   ///

   ///
   ///   true if [use AU s_ AIS]; otherwise, false.
   ///

   [Config("SQLDir")]
public string SqlDir { get; set; }
In the “Configuration.cs” file is done the reading of the attributes by reflection.
static Configuration()
{
   properties = new ConfigProperties();
   var props = typeof(ConfigProperties).GetProperties();
   foreach (PropertyInfo prop in props)
   {
       string auth = string.Empty;
       object[] attrs = prop.GetCustomAttributes(true);
       foreach (object attr in attrs)
       {
          var authAttr = attr as ConfigAttribute;
          if (authAttr != null)
          {
             string propName = prop.Name;
             auth = authAttr.Key;
          }
   try
   {
    var value = Convert.ChangeType(ConfigurationManager.AppSettings.Get(auth), prop.PropertyType);
       prop.SetValue(properties, value, null);
   }
   catch (Exception ex)
   {
    throw ex;
   }
   break;
   }
 }
}


Comment
Home

Switching between two threads in console application

Published: 9/12/2013 7:59:00 AM

I will present a very simple idea of switching between two threads. The switch can be done by user pressing the enter key or by timer.

This is an idea which I am going to use to switch between main and backup service.
In my case I am getting the data from TCP stream. It is AIS data, and if one of the streams do not receive data, should be able to switch to backup stream.

This is the example which used the user key pressing:

using System;
using System.Threading;

class Demo1
{
    /*
     * Switch between two threads. When the user presses enter key, he wakes up one of the two threads dependig on counter.
     * When the counter reaches 10, then a variable inside the threads is set to true - which indicates that the thread will finish.
     * The body of the thread is executed one more time.
     *
     *
     *
     */

    static readonly object _locker1 = new object();
    static readonly object _locker2 = new object();

    //private static EventWaitHandle _mainWait = new AutoResetEvent(false);

    static bool exit_1 = false;
    static bool exit_2 = false;

    static void Main()
    {
        new Thread(Work1).Start();
        new Thread(Work2).Start();

        int x2 = 0;

        while (true)
        {

            Console.ReadLine(); // Wait for user to hit Enter
            x2 += 1;

            Console.WriteLine(x2);

            if (x2 % 10 == 0)
            {
                //_mainWait.Set();
                lock (_locker1)
                {
                    exit_1 = true;
                    Monitor.Pulse(_locker1);
                }
                lock (_locker2)
                {
                    exit_2 = true;
                    Monitor.Pulse(_locker2);
                }

                break;
            }

            if (x2 % 2 == 0)
            {
                lock (_locker1) // Let's now wake up the thread by
                {
                    // setting _go=true and pulsing.
                    Monitor.Pulse(_locker1);
                }
            }
            else
            {
                lock (_locker2) // Let's now wake up the thread by
                {
                    // setting _go=true and pulsing.
                    Monitor.Pulse(_locker2);
                }
            }
        }

        //_mainWait.WaitOne();
    }

    static void Work1()
    {
        lock (_locker1)
        {
            while (true)
            {
                Monitor.Wait(_locker1); // Lock is released while we’re waiting
                if (!exit_1)
                    Console.WriteLine("Worker1 - Woken!!!");
                else
                    Console.WriteLine("lastly called worker1");
                //Monitor.Wait(_locker1);
                if (exit_1)
                    break;
            }
        }
    }

    static void Work2()
    {
        lock (_locker2)
        {
            while (true)
            {
                Monitor.Wait(_locker2); // Lock is released while we’re waiting
                if (!exit_2)
                    Console.WriteLine("Worker2 - Woken!!!");
                else
                    Console.WriteLine("lastly called worker2");
                //Monitor.Wait(_locker2);
                if (exit_2)
                    break;

            }
        }
    }
}

Example with automatic switch done by timer:
using System;
using System.Threading;
using System.Timers;

class Demo1
{
    /*
     * Switch between two threads. When the user presses enter key, he wakes up one of the two threads dependig on counter.
     * When the counter reaches 10, then a variable inside the threads is set to true - which indicates that the thread will finish.
     * The body of the thread is executed one more time.
     *
     *
     *
     */

    static readonly object _locker1 = new object();
    static readonly object _locker2 = new object();

    static readonly object _locker3 = new object();

    public static System.Timers.Timer HealthCheckTimer = new System.Timers.Timer();
    public static System.Timers.Timer HealthCheckTimerBackup = new System.Timers.Timer();
   
    //private static EventWaitHandle _mainWait = new AutoResetEvent(false);
   
    static bool exit_1 = false;
    static bool exit_2 = false;

    static void Main()
    {                                
        new Thread(Work1).Start();  
        new Thread(Work2).Start();
       
        int x2 = 0;
        HealthCheckTimer.Elapsed += new ElapsedEventHandler(OnElapsedTime);
        HealthCheckTimer.Interval = 100;
        HealthCheckTimer.Enabled = true;

        while (true)
        {
            lock (_locker3)
            {
                Monitor.Wait(_locker3);
            }

            x2 += 1;
           
            Console.WriteLine(x2);

            if (x2 % 10 == 0)
            {
                lock (_locker1)
                {
                    exit_1 = true;
                    Monitor.Pulse(_locker1);
                }
                lock (_locker2)
                {
                    exit_2 = true;
                    Monitor.Pulse(_locker2);  
                }

                break;
            }

            if (x2 % 2 == 0)
            {
                lock (_locker1)
                {
                    Monitor.Pulse(_locker1);
                }
            }
            else
            {
                lock (_locker2)
                {
                    Monitor.Pulse(_locker2);
                }
            }
        }
       
        //_mainWait.WaitOne();
    }

    private static void OnElapsedTime(object sender, ElapsedEventArgs e)
    {
        lock (_locker3)
        {
            Monitor.Pulse(_locker3);
        }
    }

    static void Work1()
    {
        lock (_locker1)
        {
            while (true)
            {
                Monitor.Wait(_locker1); // Lock is released while we’re waiting
                if(!exit_1)
                    Console.WriteLine("Worker1 - Woken!!!");
                else
                    Console.WriteLine("lastly called worker1");
                //Monitor.Wait(_locker1);
                if(exit_1)
                    break;
            }
        }
    }

    static void Work2()
    {
        lock (_locker2)
        {
            while (true)
            {
                Monitor.Wait(_locker2); // Lock is released while we’re waiting
                if(!exit_2)
                    Console.WriteLine("Worker2 - Woken!!!");
                else
                    Console.WriteLine("lastly called worker2");
                //Monitor.Wait(_locker2);
                if(exit_2)
                    break;
               
            }
        }
    }
}

Comment
Home

Visual Studio: the project type is not supported by this installation error

Published: 6/27/2013 4:35:00 AM

I installed ASP.NET MVC and downloaded a few example projects. I kept getting an error when opening the project file: the project type is not supported by this installation.

After some research, I found the solution over at stackoverflow: Replace theProjectTypeGuids in the csproj file to:
{349c5851-65df-11da-9384-00065b846f21};{fae04ec0-301f-11d3-bf4b-00c04f79efbc}


http://stackoverflow.com/questions/336235/the-project-type-is-not-supported-by-this-installation-error

Comment
Home

MSc Thesis in the University of Bristol - Visualising spatial and temporal electricity consumption data from Smart Meters

Published: 6/9/2013 8:14:00 AM

Since there is almost 1 year since I finished my MSc Thesis for the University of Bristol I think that it is time to share what I did with everyone interested in the topic.

Before writing the final thesis I did extensive research and got distinction result in Research Skills subject in the University. This helped me a lot for the final push when I had to research, implement and analyse the final result.

My supervisor - Prof. Peter Flach, was professor in Artificial Intelligence and unfortunately he didn't have much experience with visualization but just general knowledge. I was working with Center for Sustainable Energy (CSE) in Bristol, which have done some cool staff with heat-map visualizations. They tried to help but they were not really involved but just waited for the final output. I think that they didn't liked that I did because I used different set of technologies with which they are not so familiar with.

You can enjoy my whole thesis the way I printed. I got 68% (almost distinction) despite the negative factors which I mention above.

Comment
Home

Testing Large file upload in bleuimp Multiple file upload jQuery plugin

Published: 5/17/2013 1:58:00 PM

I had to create multiple file upload user interface for an old style asp.net web forms app. I know... old technology but what can you do when a system which is created 8 years ago is still working and you have to support it till the new version win MVC 4.0 eventually starts.
For that task I used one very nice jQuery plugin made by blueimp. It is very nice and works for most of the browsers ... who supports IE9... :) anyway in order to test the upload of large files I had create some with particular size equal to the exact maximum possible upload size.
For that reason I used cmd tool



Below is a screenshot showing an example of the execution of the program:



Comment
Home

University of Bristol - Spam Filter Assignment

Published: 1/28/2013 3:49:00 AM


Lab 5: Spam filter - Part II

In this part of the assignment, you will train your classifier on real-world e-mails, which you can download from here. They have the same convention of file-names as in part I.

Specification

The program should be written in Java, and the name of the main file should be filter.java and all files should compile using command:
javac *.java
But now, the program should take only one argument:
java filter testfile
where testfile is the name of a file containg a single e-mail, which is to be classified. The program should return the same output as in part I. Since you do not specify the directory with the training files, the program needs to store its knowledge gained during training somewhere, e.g. in a separate file (which you can submit together with your code).
The program will be tested using automatic marking on a testing set with the same number of e-mails to that in the training set, and with the same class distribution to that in the training set. The testing set will contain e-mails from the same sources as the training set. The tests will be performed on the departmental Linux machines (MVB 2.11), and your program needs to classify all testing e-mails within 30 minutes. Sample automarking script (with a single testing e-mail) can be downloaded from here.
You also need to test your program yourself, using 10-fold cross validation.
In this part of the assignment, you will discover that in many practical machine learning problems implementing the learning algorithm is often only a small part of the overall system. Thus to get a high mark from the assignment you need to implement any of the more advanced classifying techniqes or clever pre-processing methods. You will find plenty of them on the internet. If you do not know where to look for them, ask Google ;-).

Problems you might have

  1. Arithmetic underflow: Because the number of words is very large, the probabilities dealt with are very small. When multiplied together, the values may become so small they cannot be represented. The solution is to use logarithms. The formula for classification becomes:

    ci = argmax (log pi + sum (log P(wj|ci))).

    The multiplications become additions because log(a*b)=log(a)+log(b).
  2. Memory: You need to be slightly careful about how you store some information while preprocessing.

Submission

Submit your code and any files it uses via the online submission system. Also submit 2 slides that you will be using during your presentation. The presentations will take place in January in MVB 3.43, and the detailed schedule is available here. Please submit your slides in a single pdf file (NOT in Power Point native format - so if you use Power Point to prepare the slides, please print them to pdf). Please note that you will be ONLY allowed to use 2 slides during your presentation (so if you submit more, you will NOT be able to show them). Also, please use maximum 10 lines of text per slide (more lines of text will result in reduced mark), but you are welcome to include charts and tables.
Each presentation will last up to 5 minutes (if you do not finish within 5 minutes, the presentation will be interupted). The presentation should describe your work, including the methods used to preprocess the data, any simplifying assumptions made and the results of a 10-fold cross validation carried out on the data provided using your program. Do not describe the Naive Bayes algorithm. If you implement extra preprocessing options, you should report the accuracy of 10-fold cross validation with and without these options (e.g. on a slide).

Marking criteria

Below the provisional marking criteria are listed, but I reserve the right to change them.
Your program classifies the testing set with the accuracy significantly higher than chance within 30 minutes -- 30%
Your presentation cointains the results of 10-fold cross validation -- 10%
The remaining 60% of the mark will be based on the extra work you do. I attach below the criteria I used last year
- but they are likely to change this year (as last year, different weighting was given to this assignment),
so just treat them as very rough guideline.

Preprocessing
I divided preprocessing techniques used in two groups:
Simple techniques - for each simple technique I gave:
- 1% for implementation, and
- 2% for comparison of accuracy with and without the technique.
Advanced techniques - for each of advanced technique, I gave
- 2% for implementation,
- 2% for comparison of accuracy with and without, and
- 2% for analysis how the accuracy depended on parameters of a technique.

Result analysis
- analyses of different types of errors (false positive, etc) - 1%
- listing top words predicting spam/non-spam - 2%
- statistical comparison of different classifiers 1-4%

Other
Marked depending on the technique

And of course there will be 2 prizes:
- Highest accuracy on the testing set
- Most interesting approach
The winners will receive original 8'' diskettes, and some bonus points.

---------------------------------------------------------------------------------

My result for this assignment was the second best. I got 99,6% accuracy. In the report you will be able to see the techniques which I used to achieve this, but it is rally subjective - it depends of the training emails.

Presentation: http://velchev.co.uk/blogger/SpamFilter1_1_lv1594.pdf
Source Code: http://velchev.co.uk/blogger/SpamFilter1.1.zip


Comment
Home

The herding instinct...

Published: 1/4/2013 5:17:00 PM

Fascinating and amusing illustration of the tendency for people to herd or swarm. Raises questions about motivation and reward where people flock to follow or support a particular cause or product. People like to belong. Early adopters/pioneers are risk-takers. The swarm effect becomes part of the product's appeal. The internet and modern communications technologies accelerate and expand the phenomenon dramatically.

Comment
Home

String Reverse using XOR

Published: 12/28/2012 5:04:00 PM

Better implementation of string reverse is by using XOR operation. This eliminates swap operation. See how it works:


    
  public static final String reverseWithXOR(String string) {
        char[] array = string.toCharArray();
        int length = array.length;
        int half = (int) Math.floor(array.length / 2);
        for (int i = 0; i < half; i++) {
            array[i] ^= array[length - i - 1];
            array[length - i - 1] ^= array[i];
            array[i] ^= array[length - i - 1];
        }
        return String.valueOf(array);
    }

The performance of this method is very similar as the one implemented in the Java Library:

public static final String reverseWithStringBuilderBuiltinMethod(String string) {
        StringBuilder builder = new StringBuilder(string);
        return builder.reverse().toString();
    }



Reversing a string with StringBuilder built-in reverse method.
before=ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz after=zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA
Computed in 26000 ns

Reversing a string with swaps.
before=ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz after=zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA
Computed in 75000 ns

Reversing a string with XOR.
before=ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz after=zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA
Computed in 20000 ns






The source code for this article is part of the library: Java Algorithms - Implementation.

Comment
Home

Binary Search algorithm

Published: 12/21/2012 2:16:00 PM


This implementation of Binary Search algorithm is aim to be one of the most simplest. The elements are in sorted array.
If the search element (item) value is less than the middle value - > take the left part and apply the algorithm on it.

If the search element (item) value is bigger than the middle value - > take the right part and apply the algorithm on it.
If the search element is the middle element - the recursion stoops - > returns the element.
The recursion also stops if there is no such element in the array - the checks are done till start index is lest than the end index. Another interesting point is that the calculation of the middle index is done by accumulating the start index (in the beginning =0) : int index = start + (end-start)/2;
When we are somewhere in the array - the start will not be always equals to 0 as it is in the beining but will really depends on the search element and the elements in the array.




// Binary search in sorted array

public class BinarySearch
{
public static void main(String[] args)
{
int[] arr = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 
22, 34, 56, 78, 90, 112, 113, 224};

int result = find(arr, Integer.parseInt(args[0]));

System.out.println(result);
}

public static int find(int[] arr, int item)
{
return find(arr, 0, arr.length, item);
}

public static int find(int[] arr, int start, int end, int item)
{
if(arr==null)
return -1;
if(arr.length==0)
return -1;
if(start
{
int index = start + (end-start)/2;
int middle = arr[index];

if(item < middle)
return find(arr, start, index, item);
else if(item>middle)
return find(arr, index+1, end, item);
else
return index;
}
return -1;
}
}

Comment
Home

Implementation of string replace method in JAVA

Published: 12/21/2012 5:40:00 AM


The idea is to create a custom implementation of the well known method in java, which replaces substring of a string with other string.
Example:

String str1 = "Lyubomir";
System.out.println(str1.replace("Ly", "fiko"));

Result: fikoubomir

The method will be implemented using arrays or characters, although Strings can be used as well. This will be much simple, but the idea here is to get more details in how the algorithm works.

On the first pass we have to calculate how many times the string which will be replaced occurs in the input string.

private static int countNumFimesOneStrInOther(char[] str1Arr,char[] str2Arr )
{
      //count how many times the second string is in the first one
      int counter=0;
      for(int i=0; ilength
; i++){
 int j=i;
     int k=0;
     while(jlength
&& klength && str1Arr[j]==str2Arr[k])
     {         
          j++;
          k++;
     }

       //the whole string 2 is in string one
    if(k==str2Arr.length)
    {
       counter++;
    }
       }
    return counter;
}

Here for every character in the truing we check the characters in the second string, if they are the same as the ones in the first characters increment the counter. Then if the counter is equal to the size of the array - then this is occurrence.

This will give the size of the resulting character array - the size of the string - number of times the second string occurs*it's length  + number of times the second string occurs * it's length: 


char[] strResult = new char[str1Arr.length + Math.abs(numTimes*(str3Arr.length-str2Arr.length))];


Again when the whole string 2 is found in string one - iterate through the third string and put it's character by character in the result string.

Here is the full source code of the program:

import java.util.Hashtable;
import java.io.Console;
import java.util.*;
import java.io.*;
import java.lang.*;

//replace all the strings with string
public class StringReplace {
    public static void main(String[] args)
    {  
        String str1 = args[0];
        String str2 = args[1];
        String str3 = args[2];

        System.out.println(replace(str1, str2, str3));
   }

   private static int countNumFimesOneStrInOther(char[] str1Arr,char[] str2Arr )
   {
        //count how many times the second string is in the first one
        int counter=0;
        for(int i=0; ilength
; i++)         {
            int j=i;
            int k=0;
            while(jlength
&& klength && str1Arr[j]==str2Arr[k])             {
                j++;
                k++;
            }

            //the whole string 2 is in string one
            if(k==str2Arr.length)
            {
                counter++;
            }
        }
        return counter;
   }

    private static String replace(String str1, String str2, String str3)
    {
        if(str1==null || str2==null)
            return "";
      

        char[] str1Arr = str1.toCharArray();
        char[] str2Arr = str2.toCharArray();
        char[] str3Arr = str3.toCharArray();

        int numTimes = countNumFimesOneStrInOther(str1Arr, str2Arr);

        //the result string will be with length x times the number of times the string which replaces the first one occurs
        char[] strResult = new char[str1Arr.length + Math.abs(numTimes*(str3Arr.length-str2Arr.length))];

        int resIndex=0;
        for(int i=0; ilength
; i++)         {
            int j=i;
            int k=0;
            
            while(jlength
&& klength && str1Arr[j]==str2Arr[k])             {
                j++;
                k++;
            }

            //the whole string 2 is in string one
            if(k==str2Arr.length)
            {
                k=0;
                while(klength
)                 {
                    strResult[resIndex] = str3Arr[k];
                    resIndex++;
                    k++;
                    i++;
                }
            }
            
            strResult[resIndex] = str1Arr[i];
            resIndex++;
            
        }
            
        return new String(strResult);
    }
}



The complexity of the method is O(n^2) and the space needed is O(m - k.n + k.p), where n-the size of the replaced string, p-the size of the string replacement.

Better solution will be to use suffix tree or suffix array data structure - which gives the index on the substring in linear time and uses no extra space.








Comment
Home

Majority Vote Algorithm

Published: 12/13/2012 5:06:00 PM


Question: How we can determine which element in an array is majority element? (A majority element is the one which occurs most of the time compared with the other elements)

For example in the string: ABCAGRADART - A is the majority element.

A naive solution of this problem will be to traverse the string (the array) and count the number of occurrences of every character. - In that case we will store a of of unnecessary information.

Another approach is to put every element in a hash-map and then after all the elements are in the map to traverse the elements in the map and see which has the biggest value in its key/value pair.

Both of the above solutions can be used for solving the problem but there is more effective way of doing that.

If we store a pair of the current candidate for majority element and the number of occurences. When we move through the array we compare the next element with the current one...



public class MajorityVotingAlgorithm {
public static void main(String[] args) {
char[] votes = { 'A', 'A', 'A', 'C', 'C', 'C', 'B', 'B', 'C', 'C', 'C',
'B', 'C', 'C' };

int counter = 0;
char candidate = ' ';
for (char v : votes) {
if (counter == 0) {
candidate = v;
counter++;
} else {
if (candidate == v)
counter++;
else
counter--;
}
}

System.out.println("Majority Element is: " + candidate + " (counter = "
+ counter + ")");
}
}

Comment
Home

Fast, Flexible and Easy to Use N-tier Software Architecture

Published: 10/28/2010 10:16:00 PM

Introduction

The biggest aim when I developed that architecture was to be able to simplify the whole mapping process between the database and the application, but without performance losses. As we all know, the four things that can be done with a set of data are:

* The selecting of filtered data – many rows by filtering criteria
* The selecting of one row by id (unique identifier of a table), the saving of the data, this can be divided into two:
o The inserting the data in the table (for example if the id is less or equal to zero)
o The updating of the data in the table (if the id of the row is greater than zero)
* The deleting data by id

No matter how difficult the business logic is in the application, these four actions are the things that can be done and must be done with a table or a view. The next figure illustrates the N-tier architecture of the system.


http://www.codeproject.com/KB/dotnet/RDBEntitiesFramework.aspx


Let’s start with the BusinessEntities project. It is the most separated of them all but the simplest to be understood.


BusinessEntities Project

Figure 1: Class Diagram of the project BusinessEntities

The project BusinessEntities is a class library project. It’s a part of the middle layer in the N-tier architecture. It consists of container classes, which will be filled with data from the database. These kinds of classes are used only for programming representations of the data returned from the database. Usually a class like this represents a table or view in the common scenario a query which gives a list of data from the database. The name of the class corresponds to the name of the table; the name of the properties of the class corresponds to the names of the columns. These corresponding items are set by attributes. It’s done in EntityAttributes.cs file. The class MapField inherits the base .NET class Attribute. This attribute is used to map the property of the class to the column of the database. For example:
Collapse

[MapField("Title")]
public string Title
{
get { return _title; }
set { _title = value; }
}

The property called ‘Title’ of the class corresponds to the column called ‘Title’ returned from the database according to MapField attribute. The class ‘SpNames’ also inherits from the base .NET class attribute. It is used to set the names of the stored procedures for the main four actions – ‘select’, ‘select one’, ‘insert or update’, and ‘delete’. For example:
Collapse

[SPNames("pContactList", "pContactSelect", "pContactSave", "pContactDelete")]
public class ContactEntity : BaseEntity, IContactEntity
{

The constructor of the class ‘SpNames’ is overridden and if there is a need, it can be initialized with one or more of the names of the stored procedures. The way these custom attributes are used will be explained in details later. As you can see, there is a ‘BaseEntity’ class. It is used to store the two main properties in all entity classes – ‘ID’ and ‘ReturnMessage’. Every table has an id; the return message in the entity class is used to store the return message from the stored procedures. The interface ‘IBaseEntity’ is the base interface for all entities. It makes the properties ‘ID’ and ‘ReturnMessage’ to be required. It’s good if all entity classes have a corresponding interface in which all the properties can be described, but this is not compulsory.


Project DBAccess

This makes the connection with the database. It’s also a class library project. Its class diagram can be seen in Fig. 2:



Figure 2: Class Diagram of the project DBAccess

The file named Reflection.cs consists of one class called Mapper. This class, as you can guess, contains methods for mapping the entity classes and their properties with table names and columns. This is made by reading the attributes by reflection mechanism in the .NET Framework. The method ‘GetMapFields’ is one of the methods implemented in this class. By its explanation, you will be able to understand the whole reflection mechanism.
Collapse

private static void GetMapFields
(Type type, DbDataReader reader, ref Dictionary fields)
{
PropertyInfo[] pInfo = type.GetProperties();
for (int i = 0; i < attributes =" pi.GetCustomAttributes(typeof(MapField),"> 0)
{
MapField field = attributes[0] as MapField;
if (field != null && field.DbField ==
reader.GetName(i) && !fields.ContainsKey(i))
{
fields.Add(i, pi);
break;
}
}
}
}
if (type.BaseType != null && type.BaseType.IsClass)
Mapper.GetMapFields(type.BaseType, reader, ref fields);
}

The method ‘GetProperties()’ returns all properties of the type. Because the class Type is the base class of all types, here can be passed all the entity classes from the ‘BusinessEntities’ project. After the properties are returned, each property must collect its attribute defined by the developer and then they are checked if they are matched with the returned data from the ‘DbDataReader’ object. After that, the matches are saved in the Dictionary collection. By this code:
Collapse

if (type.BaseType != null && type.BaseType.IsClass)
Mapper.GetMapFields(type.BaseType, reader, ref fields);

Properties are taken as well as the properties of the base types, which are added to the Dictionary collection when there is a match. There are different implementations of the method ‘GetMapFields()’. In some are passed the type and the output collection, others the type and others the type and the object DbDataReader. The class GetEntitiesPaging will be explained in my next posting when I will say more for returning paged data. The file Filter.cs consists of ‘FilterInfo’ class, which is used for passing the parameters, when the data from the stored procedure is taken by using filtering parameters. (The filtering parameters are the parameters which limit the result or which sort the result by some criteria). The class DBAccess contains methods for creation of commands, for creation of parameters to that command and for returning of an object with type ‘DbDataReader’ and ‘DataTable’. This is the base class of the class DataTransaction. In the class ‘DataTransaction’ are made the opening and closing of the connection to the database and there are also additional methods for working with the transactions. The class ‘Paging’ stores the properties, which are necessary when there is paging of the results. The class ‘DataSource’ inherits the interfaces ICollection and IEnumberable. It contains the property ‘Paging’ which has type ‘Paging’ (the previous explained class), which is used for the paging, and also the properties for the total number of records, the list of records and the messages from the database. The class ‘Enumerator’ inherits the interface ‘IEnumerator’, there is also a property which returns the current element and methods for navigation into the collection. Another very important class is the class ‘Utils’. It contains methods which support the work of the application with the database. ‘GetDbType’: This is a method by which input type checks its name and returns the representation of the type in the database. ‘GetPropertyValue’ is a method by which input type and the name of the property returns the value of the property.
Collapse

public static DbType GetDbType(Type pType)
{
DbType ret;
switch (pType.ToString())
{
case "System.Int16":
ret = DbType.Int16;
break;
case "System.Int32":
ret = DbType.Int32;
break;
case "System.Int64":
ret = DbType.Int64;
break;

The method ‘GetProcedureName’ by input type and the type of the procedure (enum type) return the name of the procedure. This also is made by reflection mechanism. There is one nested enumerated type called ‘ProcedureType’ for all types of the stored procedures and the class EntityException. This class has as a base class - the ‘class Exception’ and it is used to throw exception when there is no defined stored procedure by the attribute of the class. The class ‘DBAccessBase’ is in the file EntityMethods.cs. It is a major class for the whole class library application. There are lots of methods for data processing, which are divided into regions.

* The ‘GetEntities’ region consists of static methods, which returns objects with type DataSource. Generics are used for passing the type of the objects. All methods are with one and the same name ‘GetEntities’ and are overriden several times depending on their input arguments.
* The ‘GetEntity’ region consists of methods which return only one object with type defined by the generalization. This returned type must be inherited by ‘IBaseEntity interface’. This means that this will be a type of the project BusinessEntities.
* The ‘SaveEntity’ region is for saving of the types from the project ‘BusinessEntities’. It’s passed object from the same type, which is not defined before – the generic mechanism is used again.
* The ‘DeleteEntity/DeleteEntities’ region contains methods for deleting of an element or many elements according to the filter.
* The Filters region contains only one static methods for adding of such filters.
* In the Other region is added code which breaks the architecture of the system, but it’s very useful in some moments. It can accept the name of the stored procedure and then execute it. This option is used when there is a need for other procedures different from the four main attributes of the entity classes. This makes the architecture not so consistent but it’s better than to create a new entity class which must derive the one which has the same type like the result from the method of the class.

Project Business

This project also like the two which I explained before is a class library project. This is a place for realization of the business rules. It consists of only one class ‘BLBase’. This class has methods, which called methods directly from the class library project ‘DBAccess’.



Figure 3: Class Diagram of the project Business made with the class designer of the Microsoft Visual Studio 2008

As you can see from the class diagram, there are methods which are very important for working with database. The method ‘DeleteEntity’ deletes one element according to the id of the element. The method ‘DeleteEntities’ deletes many elements according to filter criteria. (array of objects with type FilterInfo). The method ‘ExecuteProcedure’ executes stored procedure. The name of the stored procedure is passed as a parameter. It’s not taken like in the other methods by reflection. The method ‘GetEntities’ returns object with type DataSource. There are several implementations of that method according to the input parameters. The method GetEntitiesPaging also returns DataSource object, but with records which are determined by the input parameter of type Paging. By this parameter are passed the start record, the last record, and the count of the records. With that input data, the stored procedure will be able to return only the necessary records. The method ‘GetEntity’ returns only one element from a particular type according to an id. The method ‘SaveEntity’ saves the changes from a particular object or creates a new one according to the id property of the passed object. It returns the created object or the updated one.

Project BusinessActions

This is another layer. It’s not required but it helps because it separates the methods for returning data from the business logic of an aspx page or from the other presentation files of other projects.



Figure 4: Project BusinessActions


Project ConsoleTestApplication




Figure 5: Project ConsoleTestApplication

This is a simple console application which is used just to call methods from ‘BusinessActions Project’.
Conclusion

In my opinion, it’s very good layered architecture which is very easy to use. The only loss of performance is when the creation of the object is done. This is because reflection mechanism needs time to get properties of the class, the stored procedure name and to match them. I have made some test regarding its performance and it turns out that this only becomes a problem when a lot of objects are returned from the database, but this is not likely to happen in real life case. If you have any questions, please feel free to ask. I am open to criticism. In the next publication, I will show you a stored procedure for sorting and filtering the data. The C# code for getting data from such a procedure is written in the code, so don't worry if you don't understand it. (Example: GetEntitiesPaging procedure in the Reflection.cs file)
License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Comment
Design by: Softopia

All Content Copyright © 2013 Softopia Ltd. All Rights Reserved.